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ABSTRACT 


A  novel  approach  to  quantizer  design  through  the 
employment  of  estimation  theory  techniques  leads  to  a 
solution  to  the  problem  of  quantizing  noise  corrupted 
sources.  By  viewing  the  quantizer  as  a  general 
estimation  device,  a  risk  function  can  be  defined  for 
predetermined  cost  functions  and  minimized  to  realize  a 
quantizer  structure  which  counteracts  the  effects  of  the 
additive  noise.  The  resulting  quantizer  provides 
superior  performance  w.r.t.  the  predefined  cost  function 
over  all  other  quantizer  schemes. 

After  showing  that  any  desired  estimation  technique 
(Bayesian,  minimax,  etc.)  can  be  used,  a  constrained 
Bayesian  approach  is  used  to  develop  quantizers  for  the 
squared  error,  absolute  error,  and  uniform  error  cost 
functions.  A  specific  example,  the  problem  of  i.i.d. 
Gaussian  source  corrupted  by  additive  i.i.d.  Gaussian 
noise,  is  considered.  The  minimum  risk  quantizers  that 
result  for  the  above  cost  functions  are  presented.  It  is 
further  shown  that  as  the  noise  power  becomes  small 
(nearly  noiseless  conditions),  the  minimum  risk  quantizer 


converges  to  the  well-known  Max-Lloyd  results.  In  fact, 
in  a  noise-free  environment,  the  quantizer  requirements 
are  identical. 

The  theory  for  designing  a  minimax  risk  quantization 
scheme  is  presented  where  the  statistics  of  only  one  of 
the  sources  (clean  information  source  or  corruptive  noise 
source)  is  well  defined  and  the  other  is  uncertain.  Once 
again,  for  noiseless  conditions  the  risk  function  is 
shown  to  collapse  to  classical  results  for  both  cases. 

Results  of  quantizing  a  noisy  image  source  with 
minimum  risk  and  Max-Lloyd  quantizers  are  also  presented 
and  compared. 


TABLE  OF  CONTENTS 


APPROVAL  .  ii 

DEDICATION  .  iii 

ACKNOWLEDGEMENTS  . iv 

LIST  OF  FIGURES  .  vii 

LIST  OF  TABLES  .  viii 


Section  page 

I.  INTRODUCTION  .  1 

II.  THE  QUANTIZATION  PROBLEM  .  3 

Quantizer  Design  .  3 

Optimal  Quantization  .  7 

III.  MINIMUM  RISK  QUANTIZATION  .  11 

Estimation  Theory  .  13 

Risk  Quantization  .  15 

Squared  Error  Cost  .  20 

Absolute  Error  Cost  .  21 

Uniform  Error  Cost  .  23 

Gaussian  Source  +  Gaussian  Noise  Example .  25 

Squared  Error  Cost  .  26 

Absolute  Error  Cost  .  32 

Uniform  Error  Cost  .  38 

IV.  MINIMAX  RISK  QUANTIZATION  .  39 

Minimax  Estimation  .  40 

Minimax  Risk  Quantizer  for  Known  Source  .  41 


-  v  - 


47 


i 


Maximum  J? (  q  ,  F8,F*)  ... 

Minimax  Risk  Quantizer  . 50 

Gaussian  Source  Simulation  Example  .  56 

Minimax  Risk  Quantizer  for  Known  Noise  .  63 

V.  RESULTS  AND  CONCLUSIONS  .  69 

Minimum  Risk  Quantization  .  69 

Images  .  70 

Minimax  Risk  Quantization  .  72 

Conclusions  .  74 

BIBLIOGRAPHY  .  75 

APPENDIX  A  .  79 

APPENDIX  B  .  84 

APPENDIX  C  .  88 

APPENDIX  C  .  94 

APPENDIX  E  . 103 


-vi- 


LIST  OF  FIGURES 


Number  Deacr ipt  ion  page 

1  Quantization  Problem  .  4 

2  Classical  Quantization  .  6 

3  Additive  Noise  Problem  .  12 

4  Estimation  Problem  .  14 

5  Max-Lloyd  and  Min-Risk  Risk  Surfaces  .  30 

6  2-D  Slice  of  Risk  Surace  .  31 

7  Comparison  with  Uniform  Q.  Risk  Surface  ...  33 

8  2-D  Slice  of  Risk  Surface  (Mean  =0)  ....  34 

9  2-D  Slice  of  Risk  Surface  (Mean  =  1)  ....  35 

10  2-D  Slice  of  Risk  Surface  (Mean  =  2)  ....  36 

11  Plot  of  I  (  n)  .  51 

12  Plot  of  I  (n)  .  52 

13  Plot  of  l(n)  .  53 

14  Plot  of  I(n)  .  54 

15  Plot  of  1(0)  .  65 

16  Plot  of  1(0)  66 

17  Plot  of  1(0)  67 

18  Plot  of  1(0)  68 

19  Threshold  Condition  for  Abs-Error  Cost  ....  83 

20  Threshold  Condition  for  Unif-Error  Cost  ...  86 

21  Counter-Example  .  87 


LIST  OF  TABLES 


Number 

1 

DescriDt ion 

Comparison 

l 

o  f 

M-L 

and 

Min 

Risk  ( Sq-Er r )  ,  . 

paste 

.  28 

2 

Comparison 

of 

M-L 

and 

Min 

Risk  ( Abs-Err )  . 

.  37 

3 

Comparison 

o  f 

M-L 

M-R 

and 

Miniraax  Risk  . . . 

.  5  7 

4 

Comparison 

of 

M-L 

M-R 

and 

Minimax  ( 8dB )  . . 

.  60 

5 

Comparison 

o  f 

M-L 

M-R 

and 

Minimax  ( 4dB )  . . 

.  61 

-v  i  i  i  - 


Section  I 


INTRODUCTION 

iven  to 
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er  input 
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hat  the 

quantizer  input)  and  the  quantizer  output  is  minimum.  The 
problem  can  be  solved  by  applying  a  modified  Bayesian  Risk 
theory.  In  particular,  since  the  input  is  source  plus 
noise,  the  levels  and  thresholds  may  be  determined  such 
that  the  average  risk  is  minimized. 

An  outline  of  this  report  is  as  follows.  The 
general  quantization  problem  is  first  presented  in  Section 
II,  followed  by  a  brief  review  and  examination  of  previous 
results  in  optimal  scalar  quantization  theory.  The 


A  considerable  amount  of  attention  has  been  g 
the  quantization  problem  [  1  ] [ 3 ] [ 6 ] [ 14 ] [ 1 6 ] ,  the  re 
which  has  been  some  rather  elegant  and  useful  resu 
However,  these  optimal  quantization  efforts  have  b 
restricted  to  noiseless  sources.  When  the  quantiz 
is  that  of  a  noise  corrupted  source,  the  performan 
deteriorate.  It  will  be  shown  in  this  dissertatio 
the  effects  of  the  noise  can  be  reduced  by  conside 
quantizer  as  a  general  estimation  device.  The  pro 
then  becomes  one  of  designing  the  quantizer  such  t 
average  distortion  between  the  source  (rather  than 


problem  of  a  source  corrupted  by  independent  additive 
noise  prior  to  quantization  is  introduced  and  examined  in 
Section  III.  Also  in  Section  III,  the  theoretical 
derivation  of  an  optimal  quantization  scheme  for  this 
problem  is  presented  which  uses  estimation  theory 
techniques  to  minimize  the  risk  associated  with 
quantization  of  the  noisy  source.  This  scheme  will  assume 
prior  knowledge  of  the  source  and  noise  probability 
distributions  and  general  statistics.  The  results  are 
shown  to  converge  to  the  well  known  Max-Lloyd  results  as 
the  noise  power  decreases. 

A  minimax  estimation  approach  to  the  risk  analysis  of 
the  noisy  quantization  problem,  when  the  prior  knowledge 
is  incomplete,  is  presented  in  'Section  IV.  The  results 
indicate  a  procedure  for  quantizing  a  well-defined  random 
source  corrupted  by  noise  of  uncertain  distribution. 
Further,  the  process  is  reversible  to  yield  a  quantization 
approach  when  the  noise  distribution  is  well-defined  but 
the  source  is  not.  Once  again,  the  results  agree  with 
classical  results  for  noiseless  systems.  Finally,  results 
and  conclusions  appear  in  Section  V. 
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Section  II 


THH  QUANTIZATION  PROBLEM 

Quantization  describes  the  process  of  applying  a 
surjective  napping  of  sanples  from  a  large  domain  space  to 
a  smaller,  finite  range  space.  This  process  is 
schematically  illustrated  in  figure  1.  The  quantizer  may 
be  viewed  as  a  decision  function  incorporating  all 
decision  processes.  From  simple  scalar  threshold  logic  to 
sophisticated  multidimensional  parameter  estimation, 
quantization  provides  data  rate  reduction  and  decision 
responses  for  digital  systems.  System  performance 
criteria  necessitate  a  thorough  understanding  of  the 
underlying  concepts  of  quantization  to  obtain  suitable 
mappings . 

2 . 1  Quantizer  Design 

Most  of  the  work  in  quantizer  design  begin  with  the 
quantizer  mapping  definition  and  the  additional 
characteristic  of  input  estimation.  That  is,  a  decision 
is  made  regarding  which  region  of  the  domain  an  input 
sample  lies,  and  a  representative  value  is  determined 
which  best  estimates  elements  of  that  region.  See 
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figure  2.  Thus  possible  mappings  are  restricted  to  those 
that  provide  representative  values  which  are  elements  of 
each  respective  region,  and  the  entire  range  set  is  a 
subset  of  the  input  domain.  It  will  be  shown  that  this 
condition  is  too  restrictive  when  the  source  is  corrupted 
by  noise  prior  to  quantization. 

With  the  definition  of  a  performance  measure,  and 
possibly  additional  restrictions  regarding  prior 
knowledge,  analyses  have  been  made  which  produced 
efficient  quantizer  mappings.  In  this  work,  the  concep: 
of  quantization  as  a  combination  dec  is  ion/ es t imat i on 
device  will  be  explored  from  an  estimation  theory 
standpoint  to  provide  additional  insight  into  the 
fundamental  nature  of  quantization,  and  the  problem  of 
quantizing  a  noise-corrupted  source. 

Denote  the  quantizer  domain  space  X,  and  the  finite 
range  space  Z.  Then  the  quantizer  mapping,  y  =  q( x)  , 
produces  y  €  Z,  Vv  €  X.  The  design  of  a  quantizer 
consists  of  completely  specifying  q(;r)  and  I,  when  X  is 
known.  The  mapping  is  surjective  since  every  element  in  Z 
is  a  possible  outcome  of  q(;r),  'ix  6  X.  Since  Z  is  finite, 
call  N  the  number  of  elements  in  Z.  The  number  of 
elements  in  X  is  possibly  infinite  and  need  not  be 
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specified,  but  is  observed  greater  than  N  if  data 
compression  is  to  be  achieved.  Define  an  index  set  on  the 
elements  of  Y  so  that  Y  ={  y^  »  *  *  * ,  }  •  The  domain  can  be 

partitioned  into  N  disjoint  sets, 


Xi  ={x  |  q(x-)  =  yit  x  6  X} 

u  Xi  ■  x. 

i 


i  =1 , 


,N 


(2.1) 


As  mentioned  above,  it  is  common  practice  to  require 
that  a  representative  value  y  estimate  the  sample  x.  To 
this  end,  Y  and  q(x)  are  defined  so  that  y  closely 
approximates  x.  This  will  require  that  each  subset  X '.  be 

a  connected  subset  with  y  ^  6  X ^ .  The  most  general  notion 

of  quantization,  known  as  uniform  quantization,  partitions 
the  domain  into  N  connected  sets  of  equivalent  measure, 
with  y ^  the  center  of  X However,  it  has  been  known  for 

some  time  that  this  does  not  always  produce  the  best 
results . 

2.2.  Optimal  Quantization 

The  well  known  results  of  Max  [16J  and  Lloyd  [14J  on 
optimum  N-level  scalar  quantizer  design  have  proven  to  be 
a  good  foundation  and  basis  of  comparison  for  new 
quantization  techniques.  These  are  briefly  reviewed  for 
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purposes  of  comparison  in"  Section  Ill- 

Based  upon  the  requisite  of  input  estimation,  a 
procedure  is  outlined  for  designing  a  quantizer  subject  to 
a  general  fidelity  criterion,  t(a~x-y),  with  the  goal  of 
minimizing  the  distortion  for  an  a  priori  p.d.f.  p(  x)  on 
the  input  random  variable  X .  The  distortion  is  defined  to 
be  the  expected  value  of  f(e): 

N  l*^1 

D  -  £  f  ( x-y  *)  p(  x)  dx  (2.2) 

i=l  (I  „ 
xi 

where  the  set  . jj+1  comprise  the  partitions  (or 

thresholds)  of  the  domain  with  x^~-—  and  =  The 

partition  set  delineates  the  Xj  subsets  of  the  domain  X: 

X^  !  (  ^  i  ^2  ^  » 

xi  s  i*i>*i+ 1>*  1  <  i  S  N 

and  y ^  represents  q(  x) ,  x  £  X jf.  The  optimization  problem 

is  clearly  one  of  determining  the  partition  set  {*j}  and 

level  set  {yj}  so  that  the  minimum  distortion  is  achieved. 

The  resulting  necessary  conditions  for  optimal 
quantization  are: 


f  *  (*•  -  yd)  p(x)  dx  =  0  , 


i  -  1 ,  •  •  • ,  N 


(2.4) 


For  the  squared-error  function,  f  (  e)  =(  x-y) 2 ,  the 


expressions  become: 


yj  + 


n-i 


i  -  2  ,  •  •  • ,  N 


(2.5'. 


(  x  -  yj)  p(  x)  dx  =  0  ,  i  =  1 ,  •  •  • ,  N 


(2.6) 


For  all  but  simple  distributions,  (2.5)  and  (2.6)  are 
difficult  to  solve  analytically.  However,  optimal  scalar 
quantizers  for  several  common  densities,  such  as  uniform, 
Gaussian,  and  Laplacian,  have  been  derived.  With  the 
assumption  of  a  uniformly  distributed  X,  they  specify  the 
usual  criteria  for  a  uniform  quantizer.  For  a  Gaussian 
distributed  X,  the  quantizer  mapping  is  quite  different. 
Max  [16]  tabulated  the  results  for  a  quantizer  designed 
with  p{x)  ~  N(0,1).  The  characteristics  of  fine  division 
of  the  domain  for  small  deviation  about  the  signal  mean 
and  coarser  division  at  far  deviations  are  now  well  known. 
This  is  seen  as  a  use  of  more  quantizer  levels  for  regions 
of  high  probability  than  those  of  low  probability, 
resulting  in  smaller  error  for  the  majority  of  the  signal 
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and  lower  average  distortion.  The  superiority  of  the 
mapping  for  Gaussian  distributions  on  such  sources  is 
obvious  when  compared  to  the  uniform  mapping. 


Sect  '.on  III 


MINIMUM  RISK  QUANTIZATION 


The  problem  of  quantizing  noisy  sources,  as 
illustrated  in  figure  3,  has  yet  to  be  satisfactorially 
addressed.  An  independent  identically  distributed 
(i.i.d.)  information  source  which  has  been  corrupted  by  an 
(i.i.d.)  additive  noise  source  prior  to  quantization 
presents  a  unique  challenge  to  the  design  of  an 
appropriate  quantizer.  The  usual  quantizer  design 
procedure  yields  the  quantizer  mapping  which  produces 
range  elements  that  best  estimate  the  noisy  input  samples. 
This  can  easily  produce  unsuitable  results. 

The  problem  stems  from  the  normal  definition  of 
quantization  as  an  input  estimator.  A  modification  of 
this  definition  is  now  proposed.  By  redefining  the  goal 
of  the  quantizer  design  procedure  to  produce  a  quantizer 
mapping  which  yields  range  elements  that  best  estimate  the 
original  information  source,  the  effects  of  the  noise  can 
be  greatly  reduced.  This  extension  still  incorporates 
input  estimation  for  the  noiseless  case.  The  restiction 
of  range  elements  to  membership  of  the  domain  subset  of 
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which  the  preimage  is  a  member,  y.  €  X j,  is  no  longer 

applicable.  The  reason  for  this  is  the  noise  might 
corrupt  the  original  signal  source  such  that  the  best 
estimate  does  not  lie  within  the  observation  set  X By 

realizing  the  quantizer  problem  as  a  general  estimation 
problem,  risk  theory  techniques  can  be  applied  for  its 
solution. 


3 . 1  Estimation  Theory 


The  general  estimation  problem  is  illustrated  in  the 
state  space  representation  of  figure  4.  The  parameter 
space  0  is  related  via  a  probabilistic  transition 
mechanism  p{x\9)  to  the  observation  space  X.  The  deci.  .on 
rule  <5(at)  maps  the  observation  space  to  the  decision  space 
X.  The  problem  of  determining  the  best  estimator  y  of  9 
has  been  addressed  in  a  variety  of  ways  in  estimation 
theory.  Perhaps  one  of  the  most  well  known  is  Bayesian 
estimation.  First  a  cost  function  C(9,y)  over  0  x  J  is 
defined  which  represents  the  cost  associated  with  making 
decision  y  when  the  parameter  is  9,  based  on  observation 
x,  V  9  6  9,  x  6  X,  y  £  I .  It  is  often  appropriate  to 
express  the  cost  fuTction  as  only  the  error  between  the 
parameter  and  its  estimate. 

C(9,y)  =  C(e  =  9  -  y) 
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Estimation  Problem 


Figure  4 


Next  the  a  priori  probability  p^( 9)  is  used  with  a 
specified  cost  function  to  form  an  expression: 

R  =  E  [C  (  0,  y=6{x) )  ]  =  J  |  C  (8,y)  pQ  /  9 ,  x)  d©  dx  (3.1) 

for  the  expected  cost,  termed  the  risk.  The  Bayes 
estimate  [22]  is  said  to  be  that  which  minimizes  the  risk. 


3.2 


K 


Analogous  to  the  general  estimation  problem,  the 
quantizer  problem  for  noise  corrupted  sources  can  be 
expressed  similarly.  The  new  definition  made  here  is 
stated  as  follows:  Quantization  is  the  production  of  an 
estimate  of  a  parameter,  based  on  an  observation  of  that 
parameter,  by  a  value  from  a  lower  resolution  set  than  the 
observation  space.  It  is  easily  seen  that  if  the 
parameter  to  be  estimated  is  the  observation  (noise-free 
source  as  the  quantizer  input)  then  the  new  definition 
corresponds  to  the  usual  definition. 

It  is  desirable  to  produce  the  best  quantizer  for 
this  new  definition.  The  minimum  risk  criterion  is  now 
defined: 

R*  =  min  (  E[  C(9,y=Q(x)  ]  } 

Q(x) 
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R*  =  min  {f  f  C  (  0,  y)  p  .,(  0 ,  *•)  d  0  dx  |  (3.2) 

0(x)  L  l,X  J8  x  J 

where  Q  ( x)  corresponds  to  the  decision  rule  <5(x).  This 
criterion  can  be  examined  through  several  decision- 
theoretic  techniques,  such  as  Bayesian,  minimax,  etc., 
with  the  appropriate  restrictions,  to  produce  the  desired 
quantizer  mapping  for  that  particular  estimation  method. 
The  following  estimation  procedure  is  a  constrained 
Bayesian  estimation  problem  because  of  the  restrictions 
imposed  for  quantization.  These  restrictions  include  the 
subdivision  of  the  domain  into  disjoint  connected  subsets 
since  an  estimate  is  to  be  produced  for  each  subset  rather 
than  the  entire  domain  as  per  classical  Bayesian 
estimation.  Also  every  element  in  a  specific  subset  is 
estimated  by  a  single  value  for  that  entire  subset  instead 
of  the  infinite  range  of  values  possible  with  strictly 
Bayesian  estimation. 

Using  the  knowledge  (restriction)  that  quantization 
requires  each  value  in  the  quantizer  output  set, 

y i  6  T  ■  {  y^ ,  • • • , },  be  produced  by  an  observation 
within  the  subdivided  input  range,  x  6  [ x j • x i+ 1 1 »  the  risk 
expression  becomes: 
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f-  r  r*i+1 p  ~ 

R=  L  c««^i 

2-1  1  d*.  d-~ 


r)  P0)  *(«,  at)  d©  dA-  (3.3) 


The  mapping  6>( at)  =  <5(at)  is  then  the  choice  of  the  set  of 
quantizer  levels,  { y y)  ,  and  thresholds,  {x^},  which 
minimize  this  risk.  Now  the  risk  can  be  expressed  as: 


where : 


R  =  R^  +  J?2  +  '  ’  ‘  +  rn 


rJ>1r  ’ 

j  =  C(9,yj)  P0>  de  d*, 


(3.4) 


Vi  =  1  ,  •  •  •  ,  N 


The  cost  function  is  defined  to  be  nonnegative,  so 
the  risk  R^.  on  each  [xj’xi+\}  interval  is  always  a 
nonnegative  quant ity.  Minimizing  the  risk: 

'ik  -  !,•••, N  (3.5) 


min  * 

L  R  } 

$  min  < 

{  R  A.  } 

L  J 

{  x  A  .  {  yj) 

L  k  J 

at  a  particular  threshold  x • : 


3R  a  3 


bx 

3R 


<{ 


R1  +  R2  + 


3R. 


*  +  RN  } 
3rn 


*  T77  +  ^7+***  +  3TT  =  0 


(3.6) 


Now  the  only  intervals  affected  specifically  by  x .  are 
t^i-l.^j]  and  Afi+i  ]  ,  so: 
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pa 


3R  _ ,  »R  . 

*  ~!Tx~r  +  3T7  =  ° 


•x-j 

f  f  [  c(e'>'i-l)  Pe,x(d>x)  de  dx  } 

■*  '•U„  <*  _  oo  J 


+ 


i- 1 


r  r*i+1  "  i 

-  I  0(0,,^)  P0  ^,(0,*)  d9 

^  ^  1  v  (  —  OO  j 


(3.7) 

0 


The  joint  density  can  Le  rewritten: 

p0,  X^  01  *)  =  P^l©^!0^  Pg(s)  =  P  ©  |  9  I  *)  Pj(*) 

providing  an  expression  in  terms  of  the  probabilistic 
transition  mechanism  p^,|0(;rj0)  and  the  a  priori 

probability  of  the  parameter  space  pQ(0).  Now  for  the 
case  of  a  system  of  independent  additive  noise, 
p^|  0(*|  0)  =  p^n)  =  P  i^x-0), 
where  p^n)  is  the  a  priori  probability  density  of  the 

additive  noise  source.  For  conceptual  purposes,  most 
expressions  will  be  made  in  terms  of  the  posterior  density 
p0|^(0|jf),  with  the  knowledge  that  it  can  easily  be 

replaced  by  p ^ x- 0) p 0( 0) / p x)  for  calculation 

simplification.  Performing  the  indicated  operations  on 
equation  (3.7)  results  in  the  necessary  condition: 
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(3.8) 


P$|  x i>  dS 


C(0,rj)  P®|  y( &\xi)  d®. 


vi  =  1, 


N 


At  a  particular  level  y-i 


3R  ^ 

[  R1  +  R2  + 

*  *N  } 

3Rj 

^7 

3R9 

3Rn 

+  S77  =  0 

4 

+  *— =•  +  •  •  • 

this  implies  that  the  only  risk  defined  due  to  y  ■  is 


contained  within  R  ^ ,  therefore: 


3R. 

a 7"  =  °* 


Vi 


1, 


,N 


3 


I 

i  < 


r i+1 

r  f 

[  J 

0(9,^)  pe  ^(e,*)  de 


dx  =  0,  (3.10) 


Vi  =  1 ,  • • • ,  N 

Equations  (3.8)  and  (3.10)  together  form  the  two  necessary 
conditions  for  minimum  risk  quantization.  These 
conditions  are  applicable  for  all  choices  of  an 
appropriate  cost  function.  Further  reduction  in  the 
complexity  of  (3.8)  and  (3.10)  is  not  possible  until  after 
the  choice  of  cost  function  is  made.  Since  classically 
squared  error  distortion  is  used,  the  squared  error  cost 
function  is  selected  for  further  derivation  and  comparison 
of  the  quantizer  to  well-known  classical  results,  along 
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with  absolute  error  and  uniform  error  cost  functions. 


3.2.1  Squared  Error  Cost 


When  C  (  9-y)  -  (  9-y)  a ,  (3.8)  becomes 


d-c 


(  9-y.j-i)*  P&\y(9\Xj')  d® 


»  °° 

(  Q-y j) a  p0|  x{  9\  Xj)  de 


(3.11) 


0  de  = 


=*  I  8p4.^k)  de  =  -=* — (3.12;. 

Vi  =  1 ,  • •  • ,  N 

Using  equation  (13),  the  level  condition  for  the  squared 
error  cost  function  is: 
x , 


?>yi 


i+1 


» -< 


(e-j'j.)3  P  $  j  0  I  *)  P^*)  de  dx  =  0  (3.13; 


After  some  manipulation  this  reduces  to: 


'i+1 


»  ^  n  " 

e  P©|^e|A-)  de 

V  .  *  -so 


P^A-)  d* 


yi  = 


(3.14) 


ri+l 


A")  d  A" 


Vi  =  1 ,  • •  • ,  N 

where  the  denominator  is  the  probability  of  occurrence  of 


an  observation  within  the  set  [x-,x  Equations  (3.12) 


and  (3.14)  can  now  be  used  to  generate  the  desired 
quantizer  and  are  analogous  to  the  Max-Lloyd  expressions 
for  squared  error  distortion.  It  is  interesting  to 
examine  these  expressions  for  noiseless  conditions.  In 
this  case,  X  =  9,  and  the  squared  error  cost  is  the  same 
as  the  squared  error  distortion  of  the  Max-Lloyd 
quantizer.  The  posterior  density  P  e  |  ^  ®  I  •*)  is  a  delta 

function  at  x  and  the  mean  is  x  (likewise  the  mean  of 
PeU(©Ui)  is  x j)  .  Equations  (3.12)  and  (3.14)  then  are 

identical  to  the  Max-Lloyd  equations  for  noiseless 
condi t ions . 

3.2.2  Absolute  Error  Cost 


For  the  absolute  error  cost  function,  C ( 9-y)  =  | 9- y\ , 
equation  (3.8)  becomes: 

0  00 

J  _  \9~yi-l\  pe|  ^  9\*i>  de 

(3.15) 

| 


pe d0< 


Vi 


1,  •  •  *,N 


.ri-l 


[  (yj- 1~®)  VQ\x('&\xi )  d0  +  [  (©-y,_i)  Pg |  y(  0 1 *;)  d0 

*  —  oo  J  y  . 

^  2“1 

- 

=  (r,-©)  Pgi^^l^j)  de  +  |  (0-yj)  P0J^(  ®  I -*■_,)  de 

*  — OO  0  1/ 


.  oo 

*  }  ©  d0  =  y'ji~1 

r.  ,  0.16) 

f  i_1  r  00 

■*■  ^0"-Ki-i^  Pei  &\*i>  de  +  (9-yi)  Pqi *(  ©1*^.)  de 

j  -  «.  y  J 

If  the  posterior  density  P g | ® I is  symmetric  about  its 


mean,  it  can  be  shown  (see  Appendix  A)  that  the  mean  of 
the  density  must  lie  halfway  between  y ^  and  Y 


9  P  g  |  0  I  x  i )  d0  =  y^y21  l’  (3.17) 

Vi  =  1 , • • • , N 
but  this  condition  is  not  necessarily  true  for 

nonsymmetric  Pg | y( 0 I xj) •  For  the  absolute  error  cost 

function,  the  level  condition  is  found  to  be: 


p  g  I  yf(  0  f  *■)  P  y( 


d  9  dx 


0 


( 


» 


< 
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“  J  j  de 

i  jfj.  L  -°° 

+  [  P  ©  |  0  I  ■*■)  de 


(3.18) 

p  x)  dx  =  0 


which  reduces  to: 


", 

[/*  1 

PgU(e|*)  de 

P^(X)  dAT  =  i- 

J  Jr  . 

*J 

J  -  oo 

^  t 

,*i+l 


a-)  d*  (3.19) 


Equations  (3.16)  and  (3.19)  comprise  the  necessary 
conditions  for  a  minimum  risk  quantizer  with  an 
absolute  error  cost  function,  and  (3.17)  can  be 
substituted  for  (3.16)  when  the  posterior  density  is 
symmetric  about  the  mean. 

3.2.3  Uniform  Error  Cost 


For  the  uniform  error  cost  function: 


C (  9,  y j) 


c 


I  9-yi  I  >  A/ 2 
0  |  9-yjl  <  A/ 2 


the  threshold  condition  is: 


?i- 1_4/2  .  - 

d®  +  j  p«|  ^  ®l i') 

y  i-  i  +  A/2 

X  j“  A/  2  J  1 


de 


p  0  |  A'C  0  I  )  d0  +  P0|^(0|^j)  d9 

J  J 


(3.20) 


y  f  V  2 
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which  results  in: 


Ji~  1  +  A/2  y,-+V2 

(*  p  1 

pd|/®l^j)  d®  =  P®|^9I  xi)  d&  (3.21) 

yi~  1_A/2  yj~*/2 


For  P  9  |  ®  I  j- )  densities  which  are  continuous,  symmetric 

about  their  mean  n,  and  raonotonically  decreasing  for 
0  >  M,  then  the  condition  (3.21)  (see  Appendix  B)  can  also 
be  expressed: 

oo 

(  Q  pe|  d  9  =  1 ,  (3.22) 

Vi  =  1 ,  • • • ,  N 

The  level  condition  for  the  uniform  error  cost  function 
becomes : 


xj+l  r  yj-*/2 

Ty.  j  1  '*<’ 


+  p &  I  9 1  *)  d® 
y  j+  A/  2 


(3.23) 

P fix)  dx  -  0 


f  i+1r 

I  1  p  0 1  ^  yi~ A/  2  I  a-)  -  P9jir(/.U/2U)  ]  Pjf(x)  dx  =  0 

J*i 


or 
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(3.24) 


*i+l 

p0(rj-V2)  p^  x-{  y  f  V2) )  dx 

*  i 

*i+ 1 

P  ^  ( .r,-+ V  2  )  )  d* 

Equations  (3.21)  and  (3.24)  form  the  necessary  conditions 
for  the  uniform  error  cost  function  and  (3.22)  replaces 
(3.21)  for  the  special  conditions  on  the  posterior  density 
stated  above. 

3 . 3  Gaussian  Source  +  Gaussian  Noise  Example 

The  following  is  an  example  of  the  realization  of  a 
quantizer  satisfying  the  minimum  risk  criterion  for  a 
specific  situation,  and  is  intended  to  illustrate  the 
application  of  the  general  theory  developed  in  the 
previous  section.  Consider  a  case  similar  to  the 
Max-Lloyd  derivation  of  a  minimum  squared  error  distortion 
quantizer  for  a  source  of  known  Gaussian  statistics.  Let 
the  source  9  be  Gaussian  with  known  mean  and  variance, 

and  <Jq  respectively.  Likewise  let  the  noise  N  be  Gaussian 


i 

P9(//V2) 

I 


with  a  mean  and  variance  of  and 


3.3.1  Squared  Error  Cost  (G.S.  +  G.N.) 


For  the  squared  error  cost  function,  equations  (3.12) 
and  (3.14)  may  be  used.  Equation  (3.12)  results  in: 


where : 


K 


1 


k2  = 


Wm- 


+  a 


N 


and  p  x)  is  Gaussian  with  mean  n g+^fp  and  varian  ce  o| + 
(See  Appendix  C). 


Equations  (3.25)  and  (3.26)  are  analogous  to  the 
Max-Lloyd  equations  (2.5)  and  (2.6),  respectively.  It  is 
interesting  to  compare  the  two  sets  of  conditions  for  this 
example.  Since  the  minimum  risk  quantizer  makes  use  of 
the  additional  knowledge  of  the  noise  statistics,  it  is 
conceivable  that  the  Max-Lloyd  quantizer  could  be  designed 
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with  an  input  p.d.f.  modeled  as  the  convolution  of  the 
source  p.d.f.  and  noise  p.d.f.  With  all  of  these 
assumptions  for  this  particular  case,  the  risk  quantizer 
differs  from  the  Max-Lloyd  quantizer  through  the  values 
for  Kj  and  Rewriting, 


K 


1 


lit 


+  l 


(3.27) 


The  risk  and  Max-Lloyd  quantizers  for  this  example 
can  be  compared  by  examining  the  problem  with  large  and 
small  noise  power.  The  common  measure  of  noise  is  given 
by  the  s igna 1- t o-no i s e  ratio  (SNR),  defined  by  the  ratio 

of  the  signal  variance,  a|,  to  the  noise  variance,  o^. 

From  equation  (3.27),  it  is  seen  that  in  the  case  of  large 
SNR,  Kj  approaches  unity,  while  K2  approaches  the 

noise  mean.  This  implies  that  the  larger  the  SNR,  the 
closer  the  risk  quantizer  approaches  the  Max-Lloyd 
quantizer,  offset  by  the  noise  mean.  For  small  SNR,  is 

small  and  approaches  zero,  while  K2  approaches  Mg,  the 

signal  mean.  This  is  intuitively  pleasing,  since  it 
implies  that  the  quantizer  converges  to  the  Max-Lloyd  case 
as  expected  in  the  noiseless  case.  So  as  the  corruptive 


I 
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4-Level  Quantizers 


Gaussian  Source  +  Gaussian  Noise  Example 
(Squared-Error  Cost) 


Max-Lloyd  Minimum  Risk 


a  N 

S/N 

*i 

1 

0.  10 

20 

0 

0 . 9865 

0.4550 

1.5179 

0 

0.9865 

0.4505 

1.5029 

l 

0.25 

12 

0 

1.0118 

0.4667 

1.5567 

0 

1.0118 

0.4393 

1.4653 

l 

0.30 

10.4 

0 

1.0248 

0.4727 

1.5769 

0 

1.0248 

0.4337 

1.4467 

l 

0.  10 

20 

-2 

-1.0135 

-1.5450 

-0.4821 

-2 

-1.0135 

0.4505 

1.5029 

l 

0.25 

12 

-2 

-0.9882 

-1.5333 

-0.4431 

-2 

-0.9882 

0.4393 

1.4653 

1 

0.30 

10.4 

-2 

-1.5273 

-2 

0.4337 

-.9752  -0.4231  -0.9752  1.4467 


noise  gets  larger,  the  estimates  from  the  risk  quantizer 
approach  the  signal  mean,  while  the  estimates  from  the 
Max-Lloyd  quantizer  spread  to  cover  the  convolved  signal 
and  noise  distributions,  as  do  the  thresholds  for  both  the 
Max-Lloyd  and  risk  quantizers.  In  fact  the  thresholds  of 
the  two  quantizers  are  identical.  A  comparison  of  the 
respective  quantizer  values  at  various  SNR  rates  is  given 
in  table  1.  Note  the  compression  of  the  levels  towards 
the  signal  mean  at  lower  SNR  for  the  risk  quantizer, 
where  the  Max-Lloyd  spreads  away  from  it.  Also  note  how 
the  Max-Lloyd  quantizer  follows  the  convolved  signal  as 
the  observation  mean  changes.  The  minimum  risk,  on  the 
other  hand,  allows  only  the  threshold  set  for  the 
observation  to  shift. 

A  plot  of  the  risk  surfaces  of  4-level  Max-Lloyd  and 
minimum  risk  quantizers,  for  the  Gaussian  source  and  noise 
problem  with  squared  error  cost,  is  given  in  figures  5 
and  6.  The  Max-Lloyd  quantizer  has  been  designed  with  an 
observation  p.d.f.  of  the  convolution  of  the  source  and 
noise  p.d.f.s.  Note  that  for  zero  mean  noise,  the  two 
quantizers  are  nearly  identical  at  moderate  to  high  SNR 
values,  yet  as  the  noise  power  increases  with  respect  to 
the  signal  power,  the  risk  associated  with  each  quantizer 
diverges.  It  is  also  apparent  from  these  graphs  that  the 
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Quantizer  Risk 


gur  e 


risk  for  the  Max-Lloyd  quantizer  increases  dramatically  as 
the  magnitude  of  the  noise  mean  increases. 


For  comparison  purposes,  the  risk  surfaces  are  also 
shown  in  relation  to  the  risk  surface  for  a  uniform 
quantizer  in  figures  7  through  10.  Note  the  dramatic 
increase  in  risk  as  compared  to  the  Max-Lloyd  and  risk 
quant izers . 

3.3.2  Absolute  Error  Cost  (G.S.  +  G.N.) 


For  the  absolute  error  cost  function  the  threshold 
condition  for  the  example  is: 


Kl*i  +  K2 


*  yi-l 


Vi  =  1, 


■N 


(3.28) 


and  the  level  condition  is: 


f*i+l 

f  y±-  (K^Kg) 

erf 

/ /  „3.„3 

'v  a ga n  /  a q 

p^*)  d* 

2 


(3.29) 


p^(^r)  dx 


where  the  observation  density  p ^  x)  is  Gaussian  with  mean 
Mg+Mjy  and  variance  o|+a^.  (See  Appendix  C). 


Table  2  shows  the  results  for  the  Max-Lloyd 
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Cost  Function 


zintizer  Ris 

Note*  Mean  —  1 


Minimum  Rlok  +  Max-U^yd  o  Uniform 


TABLE  2 


4-Level  Quantizers 

Gaussian  Source  +  Gaussian  Noise  Example 
(Absolute-Error  Cost) 

M0  =  ^  =  0 


Max-Lloyd  Minimum  Risk 


S/N  (dB) 


10 

0 

0.8618 

0.3962 

1.327 

0 

0.9353 

0.3947 

1.306 

8 

0 

0.8845 

0.4068 

1.362 

0 

0.9789 

0.3921 

1.298 

4 

0 

0.9716 

0.4466 

1.496 

0 

1.  112 

0.3684 

1.222 

0 

0 

1.162 

0.5342 

1 . 790 

0 

1.360 

0.3143 

1.045 
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quantizer  and  minimum  risk  quantizer  when  the  cost 
function  is  the  absolute  error  function  between  the 
source  and  quantization. 


3.3.3  Uniform  Error  Cost  (G.S.  ±  G.N.) 


For  the  uniform  error  cost  function  the  threshold 
condition  for  the  example  is  once  again, 

-  yi  +  yi~  1 


+  H 


Vi  =  1, 


■N 


and  the  level  condition  is: 


exp(A(yj.  -  Ma)/2o|)  jerf 


’ N 


-  erf 


°N  J] 


A/  2  )  -Myyl 


(3.30) 


(3.31) 


=  exp(-A(yj.  -  Pq)  /  2  o|)  |erf  j*i  +  * 

>i-(yi-A/2)-M/Vjj 


-  erf 


The  complete  derivations  of  the  necessary  conditions 
for  the  Gaussian  source  with  Gaussian  noise  example 
presented  here  may  be  found  in  appendix  C  for  all  three 
types  of  cost  functions  considered. 
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Section  IV 


MINIMAX  RISK  QUANTIZATION 


In  the  proceeding  sections,  the  problem  of  quantizing 
a  (i.i.d.)  source  corrupted  by  (i.i.d.)  additive  noise  has 
been  examined  for  the  case  when  the  statistics  and 
probability  distributions  of  the  source  and  noise  are  well 
defined.  The  consequent  theory  developed  has  produced  an 
attractive  approach  to  quantizer  design  for  such  systems, 
and  is  equivalent  to  the  classical  result  for  noiseless 
systems.  In  this  chapter,  the  problem  of  an  additive 
noise  source  will  again  be  examined,  but  with  only 
incomplete  statistics  available  to  the  quantizer  designer 
a  priori. 

Specifically,  the  source  statistics  and  probability 
distribution  will  again  be  assumed  known  a  priori.  The 
exact  distribution  of  the  noise,  however,  will  be  assumed 
to  be  unknown,  but  belonging  to  a  generalized  moment 
constrained  class  of  distributions,  C.  A  risk  analysis  of 
the  system  will  be  applied  using  minimax  estimation  to 
produce  a  minimax  risk  quantization  scheme  for  the  system. 
Further,  it  will  be  shown  that  the  analysis  is  reversible 
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and  may  be  applied  to  the  additive  noise  problem  when  the 
noise  probability  distribution  is  well  defined,  but  the 
knowledge  of  the  source  distribution  is  incomplete. 


4 . 1  Minimax  Estimation 


It  is  often  likely  that  the  quantizer  design  must  be 
based  on  incomplete  knowledge  of  the  signal  statistics. 

For  example,  a  problem  may  arise  when  a  quantizer  is 
needed  for  a  source  where  the  statistics  are  fairly  well 
known,  but  various  unknown  noise  distributions  can  corrupt 
the  source  signal.  Likewise,  the  noisy  characteristics  of 
a  sensor  may  be  well  defined,  but  the  exact  probability 
distribution  of  the  source  signal  to  be  quantized  with  the 
sensor  unknown.  These  types  of  problems  suggest  the  use 
of  minimax  estimation  to  determine  the  quantization 
mapping.  Minimax  estimation  seeks  to  minimize  the  maximum 
possible  distortion  by  the  estimator,  and  is  directly 
applicable  to  the  risk  approach  taken  here  for  quantizer 
design . 

A 

A  minimax  estimator  t  is  one  which  guarantees  a 
maximum  risk  no  greater  than  that  for  any  other 
est imator  [ 18 ] . 

sup  R  *(0)  5  sup  R  (0)  (4.1) 

0  t*  0  t 
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Likewise,  the  minimax  quantizer  will  be  defined  as  one 
which  guarantees  the  minimum  maximal  risk  for  all 
quantizers.  Minimax  quantization  theory  has  been 
developed  for  the  classical  quantizer  problem  of  noiseless 
sources  by  Bath  and  VandeLinde  [4].  The  approach 
developed  there  will  be  applied  in  the  risk  theory 
development  of  an  appropriate  quantizer  for  the  additive 
noise  problem. 


4 . 2  Minimax  Risk  Quantizer  for  Known  Source  Statistics 


As  previously  mentioned,  it  may  be  desirable  to 
produce  a  minimax  quantization  scheme  for  the  additive 
noise  problem  where  the  source  probability  statistics  and 
density  function  are  known  a  priori  but  with  only  limited 
knowledge  of  the  noise  statistics.  Consider  the  sequence 
of  independent,  identically  distributed  random  variables 


{ 9 ,  from  the  known  cumulative  probability  distribution 

< 


function 

corrupted 

(f  XT  }  ,  tO 

Nk 


c.p.d.f)  F q.  The  source  sequence  has  been 

by  a  noise  sequence  {jV^},  with  unknown  c.p.d.f.s 

produce  the  observation  sequence  {X^}  to  the 


quantizer. 


The  sequence  { F M  },  while  unspecified,  are 
Nk 
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constrained  to  belong  to  the  set  C  of  all  possible 
c.p.d.f.s  with  whatever  a  priori  information  of  the  noise 
characteristics  is  available.  The  set  to  be  considered 
here  is  the  set  of  c.p.d.f.s  belonging  to  the  generalized 
moment  constrained  class.  These  distributions  are 
required  to  have  a  generalized  moment  less  than  or  equal 
to  some  finite  constant.  This  may  be  viewed  as  a 
restriction  of  the  noise  power  to  be  finite. 

The  minimax  risk  is  defined: 

R*  =  min  max  H(q,Fa,F.,)  (4.2) 

q€(3N  F  ysc  0  N 

where  R(q,Fg,F^)  is  the  risk  for  an  N-level  quantizer  q 
from  the  set  of  all  possible  N-level  quantizers  <3^  and  the 

c.p.d.f.s  F0I  F^  described  above.  The  minimax  risk 
♦ 

quantizer,  q  ,  is  that  quantizer  which  provides  the 

minimax  risk  R  .  The  properties  of  the  minimax  risk 
quantizer,  analogous  to  the  minimax  quantizer,  are: 

1)  V  F„e  C,  3  R  ,  >  R* 

i.e.  R*  is  the  absolute  maximal  risk  for 
quantizer  q*. 
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2 )  3  3  ^  ^3^  i  ^  max  K.  R 

FiV6C 

i.e.  q  guarantees  a  maximum  risk  no 
greater  than  the  maximum  risk 
of  any  other  quantizer  q  6  V 

The  quantizer  mapping,  q(x)  e  <3^,  maps  an  x  6  X  ■  to  a 
quantizer  level  y ^  £  Z,  i=l,***,N.  For  the  scalar 
quantizer,  X  ^  -  [  x  ^ ,  x  -+  j )  ,  where  =  -<*>,  -  °°,  and 

the  quantizer  thresholds  comprise  the  set  { x  .}  , 
i=l,  •••,N+1.  Note  that  the  set  of  quantizers  <3^  does  not 
require  the  quantizer  level  y ^  be  an  element  of  Xj.  The 

quantizers  considered  will  require  the  observation 
sequence  to  be  restricted  to  a  specific  range, 

x ^  €  t~vx,+VX^‘  T*1*3  restriction  represents  a  practical 
constraint  which  often  occurs  in  common  systems. 

The  risk  defined  in  the  previous  chapter  made  use  of 
the  a  priori  information  of  the  probability  density 
functions  p^(9)  and  p ^  n)  .  A  similar  representation  can 

be  given  here  for  the  risk  due  to  a  particular  quantizer 
mapping: 

R ( q » f e » f jy)  =  J  J  c(e,qU))  dF^pe(e)  d©  (4.3) 
e  n 

where  the  inner  integral  is  a  Lebes gue-S t i e 1 t j es  integral 
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over  the  sequence  of  noise  c.p.d.f.s 
integration  is  changed  to  yield: 

R(q,F0,Fy)  =  J  J*  C(0,q(*)) 


The  order  of 


p0(0)  d©  dF^  (4.4) 


n  e 

to  facilitate  the  ainimax  operation  procedure. 


Once  again,  it  is  apparent  that  if  the  system  is 
noiseless,  the  risk  function  is  precisely  that  of  the 
classical  quantizer  distortion  function.  The  minimax 
quantizer  then  is  the  same  as  the  Max-Lloyd  result  since 
the  source  distribution  is  assumed  known. 


R*  =  inf  P  C(9,q(x))  Pe(0)  d 9,  9  -  x 

*  0 


R*  =  inf  f  C(0,q(0))  pfl(  0)  d0 

*  i 


(4.5) 


The  cost  function  used  is  restricted  to  some  bounded 
distortion  measure,  d(0,q(x,))>  between  the  source  and 
quantizer  output: 


C(0,qU)) 


d(0,qU)), 

L, 


d(  9,  q(  x)  )  <  L 

(4.6) 

d(0,q(x-))  >  L. 


The  distortion  measure  d(0,q(x’))  is  required  to  be  even, 
continuous,  monotonic  strictly  increasing  in  |  0— <q (  at)  |  ,  and 
zero  for  perfect  estimation.  The  bound  on  the  cost 
implies  no  additional  penalty  for  a  distortion  larger  than 


% 


0 


« 


< 


( 


i 
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the  limit,  L . 


The  generalized  moment  constraint  is  given  by  some 
function  p(  n)  satisfying  the  same  conditions  as  those  for 
the  distortion  function  such  that: 

J  P(n)  dF (n)  <  c.  (4.7) 

N 

The  constraint  function  p{  n)  simply  implies  that  the  noise 
signal  power  is  bounded.  The  distortion  and  constraint 
functions  considered  here  are: 

d(9,  q(x))=\9-q(x)  |n 

(4.8) 

P(n)=\n\m  m>n>l. 

The  conditions  of  bounded  cost  and  those  imposed  on 
d(0,q(jr))  and  pin)  are  required  for  Lagrange  minimization 
of  the  next  section. 

The  term  "loading  factor"  was  given  to  the  expression 
Xfi+l/c  in  the  minimax  distortion  derivation  of  Bath  and 

VandeLinde  [4]  for  quantizers  of  noncorrupted  sources. 

This  gives  a  ratio  between  the  bounded  distortion  and  the 
input  signal  power  constraint.  Since  the  quantizer  input 
of  this  derivation  is  source  plus  noise,  the  "loading 
factor"  term  will  be  given  as: 
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1.  f . 


(4.9) 


*N+1 


(■:  *  <0 


to  reflect  the  quantizer  input  signal  power.  This  factor 
however,  will  be  unneccessary  for  the  bounded  distortion 
measure  chosen  for  this  work  since  the  bound  is  placed  by 
some  fixed  L  instead  of  the  quantizer  threshold  range. 

The  term  is  defined  here  only  for  future  work  with 
distortion  functions  similar  to  that  of  [4]. 


The  problem  then,  is  to  determine  the  minimax  risk 
quantizer  q  for  the  worst  case  c.p.d.f.,  F which 
ach i e ves : 

inf  sup  RU.Fg.F^)  =  R(q*,F0,F*).  (4.10; 

9  F  jy€C 

The  set  <3^  is  the  set  of  all  symmetric  quantizers  where 

the  levels  are  symmetric  about  the  source  mean,  and  the 
thresholds  are  symmetric  about  the  observation  mean.  The 
set  C  is  the  set  of  c.p.d.f.s  belonging  to  the  generalized 
moment  class,  and  R  is  the  risk  function  using  the  cost 
function  C(9,q(x)).  To  determine  the  minimax  risk 
quantizer,  it  first  necessary  to  find  the  maximum  risk  due 
to  a  particular  quantizer  mapping  q  €  (3^ ,  then  determine 

which  quantizer  provides  the  minimum  of  all  of  the  maximum 
risks.  The  maximum  risk  due  to  any  quantizer  mapping  q(xr) 
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may  be  determined  through  the  following  analysis. 


4.2.1  Maximum  R(q,F0,F^) 


It  has  been  shown  [4)  that  the  set  C  is  a  weak* 
compact  subset  of  a  normed  Banach  vector  space  (N.B.V.)  of 
normalized  functions  of  bounded  variation.  Also  with  any 
linear  functional  on  N.B.V.  [0,»]  (such  as  R(  q ,  F  0,  F  «)  )  , 

which  is  weak*  continuous  in  F^,  then: 

*  q  6  V  3  F'»  5  **<i>  S: 

*  *  (4.11) 

R  (q)  =  R(q,FQF^)  =  max  ^(q.Fg.F^) 

F  yV£C 

Furthermore,  a  method  for  determining  a  minimax  quantizer 
via  a  constrained  minimization  in  a  Lagrange  multiplier 

space  (Ra)  through  the  use  of  the  Lagrange  duality  theorem 
has  been  developed  [4].  The  technique  will  be 
paraphrased  here  to  determine  a  minimax  risk  quantizer. 


Let  C  =  {Fv  6  N.B.V.  [  0  ,  »]  }  where  F v  exhibits  the 


following  properties: 


1)  F ^  is  nonnegative,  monotonical ly  nondecreasing, 
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2)  J  dry  <  l, 

N 

3)  J  p(n)  F^a)  ^  c. 
N 


Define  the  convex  set  C  =  {  F^y  6  N.B.V.[0,«]  },  where  Fy 

satisfies  1).  Now  define  a  convex  functional 

G:  N.  B  .  V.  [0  ,  ~]  — ♦  Ra , 

I  "at  -  1  ' 

G  =  n 

.  J  p  djv  - c. 

which  represents  the  necessary  constraints  for  FyV  to  be  a 

c.p.d.f.  and  have  a  generalized  moment  constraint.  This 
impl ies : 

c’  =  {  F„  €  C:  GCF^)  <  0  } 


By  the  Lagrange  duality  theorem  [15],  an  expression 
for  the  maximum  risk  due  to  the  quantization  mapping  q(*) 
may  be  found: 


R*(q) 


sup  RCq.F^.F^) 


F^C 


sup  R(q,FeJF^) 


F^ec 


min 

\  ,  \  >0 
i  ’  a 

s  up 
F^C 

{  <J.  Fp>  F^ 

\  (J  dF 

-  >!  [J  '  dF/F 

(4.12) 
«=)  } 
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where  the  inner  maximization  is  achieved  for  some  worst 


L. 


R 

R 


i 


*  — 

case  distribution,  F ^  6  C,  and  the  outer  minimization  by 

X A  and  Xa.  If  the  cost  weighting  function  is  bounded, 

R*(q)  is  finite  and  equation  (4.4)  can  be  used  to  replace 
Rtq.F^F^)  of  equation  (4.12)  to  yield: 

R*(q)  =  min  [(X  +  X  c)  +  B(X  X  )]  (4.13) 

X1,x2>0  1  2  12 

where 


B(Xt, xa) 


max 

F^eC 


J  I(j»)  dF  jy(  n) 
ff 


(4.14) 


I(n)  =  J  C  (0  -  q(*))  Pe(0)  de  -  Xt  -  Xap(n)  (4.15) 

e 


The  minimization  of  equation  (4.13)  need  only  be 
considered  when  B(X^,X  )  is  finite.  This  will  only  occur 

when  I(n)  is  nonpositive.  If  I(n)  is  positive  at  any 
point,  a  sequence  of  noise  random  variables  { JVY,}  with 


c.p.d.f.s  {F^} 
Jc 


could  occur  such  that  as  k  gets  large, 


the 


integration  tends  to  infinity. 


If  I ( n)  is  required  to  be  nonpositive,  then  the 
maximization  of  equation  (4.14)  over  all  possible  F^  6  C 


-49- 


imp  1 ies : 


BU^X^)  =  0. 


(4.16) 


Since 


constant  is  an  element  of  C  for  { JV^}  , 


i  t 


follows  that  if  I  ( /7)  is  made  nonpositive,  the  maximum 
possible  integration  of  equation  (4.14)  is  zero. 


The  minimax  risk  for  any  quantizer  q  6(3^  and  the 
specified  conditions  then  reduces  to: 

R*(q)  =  min  (X  +  X  c)  (4.17) 

X  ,  X  >0  1  a 

i  ’  a 

I  (a) <0 


A  series  of  plots  appear  in  figures  11  through  14 
which  depict  the  function  I  ( a)  for  various  quantizers  with 
Pe( 0)  ~  N  (  0 , 1 )  ,  and  distortion  bounds  of  L  =  0.25,  0.75. 

These  show  the  effects  of  the  distortion  bound  on  the 
function,  and  that  it  is  quantizer  dependent.  Note  the 
discontinuity  near  the  quantizer  threshold. 

4.2.2  Minimax  Risk  Quantizer 


£ 

The  minimax  risk  quantizer  q  then,  is  that  quantizer 
which  produces: 
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NOISE 


I 
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R*  =  min  min  (  X  +  X  c )  (4.18) 

qetfN  VX3-°  1  2 

I(n)  <0 


This  result  is  similar  to  that  obtained  for  minimax 
quantization  by  Bath  and  VandeLinde  for  the  generalized 
moment  constrained  class  of  quantizer  input  distributions. 
It  differs  primarily  in  the  function  I(o)  which 
incorporates  the  cost  of  estimating  the  source  by  the 
quantization  mapping  and  the  a  priori  knowledge  of  the 
clean  source  probability  distribution  function,  F  Q(  9)  . 


The  minimization  of  equation  (4.13)  over  all 
nonnegative  X^,  Xa,  and  nonpositive  I ( o)  implies  the 

maximum  of  I(/i)  is  zero.  As  can  be  seen  in  equation 
(4.15),  the  role  played  by  in  I ( n)  is  merely  that  of  a 

bias  term.  This  implies  that  the  minimization  of  equation 
(4.13)  can  be  determined  by  solving: 


\(xa)  =  max 


ax  I* 
a  L  a 


C  (  9,  q(A’)  )  Pfl(e)  d 9  -  \_p(n) 


(4.19) 


and  minimizing  over  all  possible  Xa  >  0.  Maximization  of 


I(n)  with  respect  to  Xa  and  X*(Xa)  for  a  specific 
quantizer  q  6  (3^  will  then  yield  the  minimum  Xg  and  the 
requisite  X  for  max  I ( n)  =  0. 
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The  minimax  risk  quantizer  determination  procedure, 
while  quite  numerically  tedious,  may  be  described  by  the 
fol lowing: 

1)  Select  X2  >  0. 

2)  Determine  the  optimal  X  for  the  specified 

X2:  X*(X2). 

3)  Repeat  step  2)  over  all  X2  >  0  to  determine 

* 

the  minimum  of  (X^(X2)  +  X2  c) .  This  is  the 

♦ 

maximum  risk  for  the  quantizer  q:  R  (q). 

4)  Repeat  steps  l)-3)  over  all  quantizers 
q  6  <JN  to  determine  the  minimum  of  the 

maximum  risks:  R  =  min  R  (q). 

5)  The  minimax  risk  quantizer  is  that 
quantizer  q*  which  yields  the  minimax  risk: 

R*(q*)  =  R*. 


Table  3  lists  the  results  of  the  minimax  risk 
quantization  simulation  compared  to  the  Max-Lloyd  and 
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TABLE  3 


4-Level  Quantizers 

Gaussian  Source  +  Gaussian  Noise  Example 
(Squared-Error  Cost) 
^0=^=0 


Max-Lloyd  Minimum  Risk  Minimax-Risk* 

S/N  (dB)  x  i _ yj _ xj _ y^ _ x  i  y.  c 


8 

0 

0.4874 

0 

0.4774 

0 

0.49 

0.  16 

1.056 

1.625 

1.056 

1.298 

1.2 

1.25 

4 

0 

0.5354 

0 

0.383 

0 

0.  15 

0.4 

1.1604 

1.7854 

1.1604 

1.277 

1.1 

1.0 

0 

0 

0.6404 

0 

0.3202 

0 

0.  13 

1.0 

1.388 

2.136 

1.388 

1.0678 

1.4 

1.3 

% 

Minimax-Risk  quantizer  calculated  with:  L  =  0.25 

P9(&)  ~  N(  0 , 1 ) 

steps  ize  =  *-*- 
25 
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minimum  risk  quantizers  for  a  Gaussian  source  with  the 
squared  error  cost  function.  The  minimum  risk  quantizer 
assumes  the  noise  to  also  be  Gaussian.  While  the 
simulation  results  for  the  minimax  risk  quantizer  appear 
somewhat  ambiguous,  there  are  several  reasons  which  offer 
plausible  explanation.  First,  the  calculations  involved 
in  the  Lagrange  minimization  are  enormous  and  the 

4 

precision  in  the  calculation  could  easily  influence  the 
outcome  of  the  calculated  maximum  risk  for  any  given 
quantizer.  Second,  the  precision  of  integration  over  the 
known  density  function  can  further  influence  the  risk 
calculation.  For  the  results  presented  here,  the 
integration  was  approximated  by  a  Riemann  sum  of  25 
samples  of  the  density  function  over  [m0-2oq,  m0+2 o Q] , 

which  can  further  affect  the  risk.  Third,  the 
determination  of  the  maximum  of  l(n),  over  all  possible  N, 
due  to  computational  constraints,  must  be  limited  to  a 

sampled  range.  The  range  used  here  was  yV  6  [  ±  2  /cj, 
sampled  in  50  equivalent  intervals.  As  illustrated  in  the 
plots  of  I ( n) ,  this  may  cause  some  error  in  determining 
the  maximum  if  the  samples  miss  the  region  of  the 
discontinuity  in  the  function.  And  finally,  since  these 
calculations  must  be  performed  over  "all"  N-level 
quantizers  (not  limited  to  levels  appearing  within 
specific  ranges  as  in  classical  quantization),  the  step 
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size  imposed  may  effectively  predetermine  a  suboptimal 
solution.  The  quantizer  step  size  used  here  was  the  same 
as  the  step  over  the  noise  range. 


t 


I 


It  appears  from  the  plots  of  I ( n)  that  the  areas  of 
maxima  occur  at  or  very  near  the  quantizer  thresholds. 
While  it  will  not  be  shown  here  that  this  is  the  case,  a 
simulation  has  been  performed  based  on  this  assumption 
with  the  results  appearing  in  tables  4  and  5.  These 
results  were  determined  by  sampling  I ( n)  in  the  region 
surrounding  each  threshold,  with  greater  accuracy  imposed 
in  all  calculations  than  those  in  the  previous  simulation. 
Specifically,  the  quantizer  step  size  was  fixed  at  0.01, 
the  number  of  intervals  in  the  Riemann  sum  was  increased, 

and  the  accuracy  of  the  Lagrange  minimization  was 

_  n 

increased  to  1  x  10  .  The  minimax  risk  quantizer  with 

specific  quantizer  thresholds  were  found,  and  the  minimum 
of  these  selected  as  the  minimax  risk  quantizer. 


* 


i 


Comparisons  of  the  minimax  risk  quantizers  to  the 
Max-Lloyd  and  minimum  risk  quantizers  (G.S.+G.N.)  are 

shown  at  the  bottom  of  tables  4  and  5.  At  8  dB ,  the 
thresholds  of  all  three  are  nearly  identical,  with  the 
levels  for  the  minimum  risk  quantizer  inside  (closer  to 
the  mean)  those  of  the  Max-Lloyd  quantizer,  and  the 
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TABLE  4 


I 


4 

1 

Minimax  Quantizer  Determination  1 


(N  =  4, 

N  ~  °* 

*1 

Squared- 

L  =  0.25, 

*0 

Error  Cost, 

SNR  =  8dB 

yi 

P Q  ~  N(0,  1)  ) 

=*  c  =  0. 1585 

Minimax-Risk 

0.90 

0.41 

1.23 

0.09931 

0.91 

0.41 

1 . 24 

0.09854 

0.92 

0.41 

1 . 24 

0 . 09780 

0. 93 

0.41 

1.24 

0.09708 

0.94 

0.41 

1 .  24 

0.09638 

-*  0.95 

0.42 

1.24 

0.09571  < - 

0.96 

0 . 42 

1 .  28 

0 . 09743 

0 . 97 

0.42 

1.31 

0.09921 

0.98 

0 .41 

1.31 

0 . 09875 

0.99 

0.41 

1.31 

0.09818 

1.00 

0.41 

1.31 

0 . 09762 

1.01 

0.41 

1.32 

0 . 09709 

1.02 

0.41 

1.32 

0 . 09657 

1.03 

0.41 

1.32 

0.09606 

1 . 04 

0.41 

1 . 36 

0.09807 

1.05 

0.41 

1.39 

0.09973 

Max- 

Lloyd 

% 

Minimum  Risk 

Minimax 

Risk 

*i 

yi 

xi 

yi 

xi 

yi 

0 

0.4874 

0 

0 . 4774 

0 

0.42 

1 .  056 

1 . 625 

1 . 056 

1  .  298 

0 . 95 

1 . 24 

♦ 

Minimum  Risk  for  Gaussian  Sources. 


I 
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TABLE  5 


Minimax  Quantizer  Determination 


(N  =  4 

,  Squared- 

Error  Cost, 

P Q  ~  N(0, 1) ) 

~  0  * 

L  =  0.25, 

SNR  =  4dB 

$  c  =  0.398 

*1 

yo 

y\ 

Minimax-Risk 

1.27 

0.41 

1.51 

0 .  12232 

1.28 

0 .41 

1.53 

0 . 12372 

1.29 

0.41 

1 .56 

0 . 12500 

1.30 

0.41 

1.56 

0 . 12447 

1.31 

0.41 

1.56 

0.12395 

1.32 

0.41 

1.57 

0 . 12343 

1.33 

0.41 

1.57 

0. 12294 

1.34 

0.41 

1.57 

0 .  12254 

— ►  1.35 

0.41 

1.57 

0.12198  < - 

1.36 

0.41 

1.58 

0. 12348 

1.37 

0.41 

1.60 

0 . 12492 

1.38 

0.41 
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minimax  risk  quantizer  levels  just  slightly  inside  those 
of  the  minimum  risk  quantizer.  At  4  dB ,  the  outer 
threshold  of  the  minimax  risk  quantizer  has  migrated  well 
outside  of  the  other  two  quantizers,  which  may  be 
from  the  Max-Lloyd  and  minimum  risk  quantizers  assumption 
of  Gaussian  densities,  while  the  minimax  risk  quantizer 
only  considers  constrained  noise  power.  The  levels  of 
the  minimax  risk  quantizer  are  now  between  those  of  the 
Max-Lloyd  and  minimum  risk  quantizers.  This  may  be 
interpreted  from  the  standpoint  of  the  assumptions  also. 
The  minimum  risk  quantizer  assumes  both  the  source  and 
noise  are  Gaussian,  so  it  arrives  at  closely  compacted 
estimates  for  the  levels,  while  the  minimax  risk 
quantizer  only  assumes  the  source  is  Gaussian  for  this 
example,  and  the  noise  power  constrained  —  so  its 
estimates  are  outside  those  of  the  minimum  risk 
quantizer  (for  this  example).  Compared  to  the  Max-Lloyd 
quantizer,  the  minimax  risk  quantizer  is  "aware"  of  the 
corrupting  influence  of  the  noise,  whereas  the  Max-Lloyd 
is  not  and  the  estimates  for  the  quantizer  levels  of  the 
minimax  risk  quantizer  are  then  placed  within  those  of 
the  Max-Lloyd  quantizer. 


fl 


« 


4 


i 


( 
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4 . 3  Minimax  Risk  Quantizer  for  Known  Noise  Statistics 


The  minimax  risk  theory  just  applied  to  the  problem 
of  quantizing  an  i.i.d.  source,  with  known  probability 
distribution,  corrupted  by  a  sequence  of  independent 
additive  noise  random  variables  each  of  uncertain 
distribution,  is  easily  modified  for  the  converse  problem 
If  the  noise  sequence  is  i.i.d.  with  known  c.p.d.f.  F^, 

but  the  source  sequence  {  0^.}  is  only  known  to  be 


distributed  with  the  c.p.d.f. 


sequence 


(F0  } 

9k 


from  the  set 


C,  then  the  risk  may  be  written: 


^(q.Fg.F^)  =  J  J  C(0,qU))  p J^n)  dn  dF Q  (4.20 
0  N 


The  minimax  risk  quantizer  development  for  this 
problem  parallels  the  development  for  the  previous  case. 

A  similar  solution  is  described  as  the  minimax  risk 

quantizer  q  which  yields  the  minimum  maximal  risk, 

R*  ( q ) ,  where : 

R*(q)  =  min  (  X  +  X  c )  (4.21; 

X  .  X  >0  1  2 

1  ’  2 

I(  9) <0 

and 

1(0)  =  J  C(0  -  qU))  P j/n)  dn  -  xt  -  x2p(0)  (4.22; 

N 


-63- 


The  same  restrictions  of  the  previous  section  apply  to 
the  distortion  and  constraint  functions,  d(0,q(*O)  and 
p(  9)  .  Likewise, 

X*(Xfl)  ~  max  J  C(0,q(x>)  p^fl)  d/»  -  \ap(6)  (4.23) 

9  *•  N  J 

and  the  minimax  risk  quantizer  q  is  specified  by  a 
procedure  analogous  to  that  of  the  previous  section. 

Figures  15  through  18  show  the  effects  due  to  the 
constraint  on  the  noise  power  and  the  distortion  bound  L 
on  the  function  1(0).  These  appear  roughly  similar  to 
I(s)  in  [4],  differing  through  the  knowledge  of  the  noise 
for  1(9). 

Note  that  in  the  noiseless  case,  this  risk 
definition  coincides  precisely  with  the  distortion 
function  of  an  unknown  input  sequence  used  by  Bath  and 
VandeLinde.  The  minimax  risk  quantizer  then  is  identical 
to  the  minimax  quantizer  developed  there  for  the  classical 
quantizer  distortion  measure. 
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PLOT  OF  I (THETA) 


Section  V 


RESULTS  AND  CONCLUSIONS 


A  quantizer  which  coapensates  for  noise  should  be 
quite  useful  in  signal  processing  systems.  The  theory 
developed  for  the  quantizer  design  is  unique  and  presents 
a  new  way  of  thinking  about  quantization  by  modifying  the 
normal  definition  of  the  quantizer  as  an  input  estimator 
to  a  general  estimation  device.  The  definition  of  a  risk 
associated  with  an  estimation  of  the  source  by  the 
quantizer  is  introduced  to  provide  a  means  for  determining 
the  specific  quantizer  mapping.  Achieving  a  minimum  risk 
criterion  is  the  goal  of  whichever  estimation  technique 
is  employed. 

5. 1  Minimum  Risk  Quantization 


For  the  case  where  the  source  to  be  estimated  is  the 
quantizer  input,  the  extended  quantizer  definition  reduces 
to  the  traditional  definition  [16],  and  the  development  of 
the  minimum  risk  quantizer  parallels  that  of  the  optimal 
distortion  development.  Necessary  conditions  for 
minimum'  risk  quantization  have  been  derived  and  specific 


expressions  for  the  squared  error,  absolute  error,  and 
uniform  error  cost  functions  have  been  developed.  For  the 
specific  example  of  Gaussian  source  plus  Gaussian  noise, 
quantizers  were  derived  with  these  three  cost  functions, 
and  tables  generated  for  the  squared  error  and 
absolute  error  cost  functions  at  various  signal-to-noise 
rates. 


Quantizer  performance  is  normally  measured  with 
respect  to  the  quantizer  distortion  (based  on  the  input). 
For  the  class  of  risk  quantizers  developed  here,  this 
would  have  no  meaning  since  the  object  of  risk 
quantization  is  that  of  original  source  estimation  and  not 
the  noisy  input.  The  quantizers  must  be  compared  then  to 
the  accuracy  of  estimation  of  the  source,  which  is 
provided  by  the  risk  function  itself. 


5 . 2  Images 


While  the  risk  surfaces  show  the  respective 
performance  in  terms  of  signal  variance  to  noise  variance 
ratio  (SNR)  for  various  noise  statistics,  as  a  more 
graphic  demonstration  of  the  quantizer  performances, 
sample  images  were  quantized  and  compared.  Some  of  these 


images  appear  in  appendix  D  and  illustrate  the  basic 


properties  of  the  minimum  risk  quantizer. 

The  images  presented  show  the  effects  of  source 
quantization  with  Max-Lloyd  and  minimum  risk  quantizers 
for  noise-corrupted  sources.  The  source  considered,  a 
picure  element  (pixel)  of  a  digital  image,  represents  the 
light  intensity  of  a  minute  area  of  a  monochromatic 
picture.  The  source  is  assumed  Gaussian  and  the  squared 
error  cost  function  is  used  to  design  the  quantizers.  The 
statistics  for  the  source  were  estimated  by  the  statistics 
of  a  sampled  window  about  the  face  of  the  person  in  the 
image . 

While  it  is  realized  that  the  assumption  of 
independent  Gaussian  statistics  for  the  image  is  a  crude 
approximation  to  the  actual  situation,  the  images 
demonstrate  the  superiority  of  the  minimum  risk 
quantization  to  that  of  Max-Lloyd  and  uniform 
quantization.  They  further  illustrate  some  of  the  basic 
properties  of  all  the  quantizers.  Note  particularly  how 
the  Max-Lloyd  quantizer  follows  the  observation  mean  when 
the  noise  is  caused  to  offset  the  mean  of  the  image, 
whereas  the  risk  quantizer  compensates  accordingly.  The 
original  image  and  the  noisy  images  are  repre  sented  by 
256-level  gray  scale,  while  all  remaining  images  were 


quantized  with  4-level  quantizers.  The  images  (appendix 
D)  are  presented  in  the  following  format: 


Original  Image 
Source 

256  Levels 

Max-Lloyd 

Original 

4  Level 

Max-L 1 oyd 
Orig+Noise 

4  Level 

Noisy  Source 
256  Levels 

Uniform 

4  Level 

Min.  Risk 

4  Level 

where  "Original"  refers  to  a  Max-Lloyd  quantizer  designed 
solely  for  the  clean  source  image  and  "Orig+Noise"  refers 
to  one  with  is  designed  for  the  observation  image  of 
source  +  noise. 


Sample  images  were  also  examined  for  the  absolute 
error  cost  function,  and  the  results  were  similar  to  those 
for  the  squared  error  cost  function.  A  comparison  of  the 
subjective  image  quality  between  the  two  cost  measures  did 
not  yield  a  significant  difference  between  them,  so  sample 
photos  for  the  absolute  error  cost  function  are  not 
included  in  this  dissertation. 


5 . 3  Minimax-Risk  Quantization 


The  risk  functions  examined  for  the  minimax  risk 
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quantization  indicate  the  ability  to  quantize  a  source 
when  the  source  statistics  and  probability  density 
function  are  well-defined,  but  the  noise  probability 
density  function  is  known  to  belong  only  to  a  general 
class  of  density  functions  (the  generalized  moment 
constrained  class).  The  process  has  also  been  shown 
reversible  and  may  be  applied  to  the  problem  when 
well-defined  noise  statistics  and  distribution  (such  as 
specific  characteristics  of  a  sensor,  or  camera  effects 
are  imposed  on  the  quantization  process,  but  the  source 
distribution  is  primarily  unknown. 

Both  of  these  situations  are  based  upon  the  risk 
function  definition,  which  as  the  noise  power  decreases, 
converge  to  well-known  classical  results.  Specifically, 
for  the  known  source  corrupted  by  unknown  noise,  as  the 
noise  power  approaches  zero,  the  risk  function  becomes 
precisely  that  of  the  quantizer  distortion  of  Max  [16]. 
Likewise,  when  the  noise  statistics  are  well-defined,  but 
source  is  unknown,  as  the  problem  approaches  the  noiseless 
case,  the  risk  function  becomes  the  quantizer  distortion 
used  by  Bath  [4]  for  the  minimax  quantizer  distortion. 
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5 . 4  Conclusions 


The  theory  and  results  presented  here,  using  a  new 
risk  theory  approach  to  quantizer  design,  demonstrate 
significant  improvement  over  classical  designs  for  noisy 
sources.  For  independent  additive  noise,  the 
quantization  problem  has  been  formulated  and  solved,  both 
for  well-known  noise  distributions  and  uncertain 
distributions.  Also,  when  the  source  statistics  are 
uncertain  but  the  noise  is  well-defined,  a  minimax  risk 
quantization  may  be  achieved. 


i 


The  risk  theory  approach  suggests  quantization 
schemes  for  other  problem-,,  such  as  correlated  samples  or 
multiplicative  noise,  may  now  be  developed.  Also,  the 

approach  appears  extendable  to  the  problem  of  * 

multidimensional  quantization,  and  suggests  possible 
advancements  in  the  area.  Further,  the  same  techniques 

employed  for  the  quantizer  design  development  shows  the  < 

promise  of  the  risk  theory  approach  to  other  applications 
as  well  as  that  of  quantization,  such  as  neural  network 

theory .  i 
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APPENDIX  A 


The  threshold  condition  for  the  absolute  error  cost 
function  was  given  in  (3.16)  as: 


9 


X-)  d9  = 


y?yj-\ 


/i- 1 


(A.  1) 


Pe  |  a'C  9 1  *i>  d9  +  Pq\^9\ x  d9 


where  { y2)  ,  i= 1 . N,  are  ordered  y  ■  ,  <  y^.  Define 


9  p  9  |  9  *  xi  ^  d  9 


(A. 2) 


for  all  i.  Now  if  P$ | ^ 0 1 )  is  symmetric  about  n then 


9  p  9  |  9  I  *  i  ^  d  9  = 


yj+yj-i 

2 


Vi  =  1 ,  •  •  ■ ,  N  (A. 3) 


* 


f 


a 


i 


Proof 

t 


Case  u  <  iXhzl 
1  2 

By  symmetry: 
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(A. 4) 


[  P0|^0l^i)  d® 

yi 

2>1i~yi 

-  -  j  (©-(2 de 

*  -  oo 

2m  J r-y.- 

y  ,+y  ,_i  I* 

Mi  =  - g -  +  J-(2MJ-yi-yi_i)  P9U(ei^)  de 

fri-l 

+  (s-y^-i)  Peij(elA'i)  de 

2  m  •-  y  " s 

*  i 


It  is  desired  to  determine  all  possible  such  that  (A. 5) 

y  ■+  y  , 

exists.  At  m,  =  — - - - - ,  the  last  two  terms  on  the  R.H.S. 

1  2 

are  zero  and  the  expression  is  satisfied.  Now  consider 


(A. 5) 


(31)  as  a  condition  of: 


f(Mj)  =  g(  **_,•) 


(A. 6) 


where  f  ( n  j)  =  (L.H.S.  of  (A. 5))  and  g  (  m  ^- )  =  R.H.S.  of 

(A. 5).  It  is  necessary  to  determine  all  such  that 
(A. 6)  exists.  It  has  been  shown  that  f(M^)  =  g(/^)  for 

y  .+  y  .  , 

Mi  =  — - — — ,  so  it  is  necessary  to  examine  (A. 6)  for  all 

1  2 

M i  <  •  Taking  the  derivative  of  each  function  w.r.t. 
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(A. 7) 


f  *  =  1 


&'  i  P  j)  = 


„2tli~yi 

2  p  9  |  AT(  9  1 


de 


by  Leibnitz’  rule.  For  ai  .  <  and  <  y^  4 

^  Mj  <  y_j-  ^  2/ii~yi  <  Mi‘  In  this  case  g’(Mj) 
less  than  1,  approaching  it  as  Mj  •  Since  it 

shown  that  the  two  functions  intersect  at  Mi  i  and 
first  derivatives  are  never  equal  at  any  other 
region  of  interest)  this  implies  the  intersection 
unique.  See  figure  19. 


Case  2)  /*•  2 


>  yi+yi~  1 


The  argument  parallels  that  for  case  1  with: 
f"- 1 

J  (^i-i)  Peu(0l*i>  d® 


(e-(2Mi-yi_1))  Pei/eUj) 

2Mi_yi-l 
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"i  <  J'i 
is  always 

has  been 

their 

(for  the 

at  is 


(A. 8) 

de 


Figure  19 
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APPENDIX  B 


The  threshold  condition  for  the  uniform  cost  function 

was  : 


.k_i  +  a/2  x,-+a/2 

l  -Z  1  p 

p9|^(0i  xii  de  =  p  Q\x^6\xj)  de  (B.I) 

yi-  1~A/2  y  i~  A/  2 

Now  if  P q | 9 I xi )  continuous»  symmetric  about  mean  Mj, 

and  monotonic  decreasing  for  9  >  m ^ ,  it  can  be  shown  that 

it  is  sufficient  to  say: 


9  pe|/0l*j)  d9  = 


LlZa-1 


Vi  =  1, 


i  N  ( B  .  2  ) 


See  figure  20.  By  inspection,  V  A  3  y^ ,  ^  where 


< 

yl  < 

y2  ’ 

3  (B.I)  holds. 

Likewise,  3 

yl  ’  y2  where 

yl  < 

y2  < 

"i’ 

for  which  (B.I) 

holds.  And 

it  easy  to 

see  that  for  fixed  A  and  y ^  >  (B.I)  holds  only 


-  yl  +  y2 

for  M  ,•  =  — - -.  A 

2  2 


of  a  nondecreasing, 


counter-example  is  given  in  figure  21 
but  symmetric  P q | ® I x j )  • 


t 


i 


» 
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APPENDIX  C 


The  following  is  the  mathematical  derivation  used  by 
the  Gaussian  source  corrupted  by  Gaussian  noise  example  of 

chapter  III  for  the  minimum-risk  quantizer.  If  the  source 

9  ~  N(p0,o0)  and  noise  N  ~  N  ( Py,  Oy)  are  independent, 

X  =  9  +  N  =*  Px(x)  =  p  Q(x)  0  Pf/.x) 

( C  .  1 ) 

Pj^x-9)  p &(  9)  d  9 

Since  the  sum  of  two  independent  Gaussian  random  variables 

is  Gaussian,  then  p x)  ~  N (  )• 

C . 1 .  Squared  Error  Cost 


For  this  example  and  the  squared  error  cost  function, 
equation  (3.14)  becomes: 


»  -*‘+l  * 

I  9 

^  V  .  —  oo 


x-9)  pa(9)  d  0 


d* 


*i+l 


(C.  2) 


J 

x  i 


dx 


Vi  =  1 ,  • •  • ,  N 
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Define : 


g{x)  =  J  0  p  jy(  x~  9)  p0(0)  d0 


(C.3) 


-  f  "  9  ..J  if  al(^0-Va  +  aVe-fi9>*  11 


d  9 


Expanding : 

®  0  (  Af-  9-  /I yy)  +  O  9-  M  0)  =  (  O  0+  O  y)  X  +  (  —  2  0  Q^r+  2  (J  pf~  2  yyA*  0)  AT 

+  a|^a  -  2c§m^  +  +  a^l 


=  (a%*a%) 


Define : 


0  - 


a9x~a9flU+aJ^9 


~°9X+  «%»N~aV  9  1 

a|+cra  J  l  <*1+^ 

g0+gAT 


„  2 ,  _  2 
a  9+a  fif 
„2  2  * 
g0g/V 


-  g9'y~g9M/V*gA'*i9 
g0+g/V 


s3  +  g9Ara~2g9>iyr+g9^a+g>0 
'  _  2  , _ 2 


3,  ^2.,  2 


C3  =  ~^C2 
Then  g(x)  becomes: 


g9+gAr 


-  O©  ^ 

=  J  2n<J9~(jjy  exP  {  -  2-  Cl[t*  -  C2]a*  C3  }  de  (C-4> 

w  —  oo 
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g(  x)  - 


exp 


r  -c ic3  1 

r  00 

\  ~T~  / 

'V  a  tf  « 

—  00 

L  5/2  IT  0^0^/  (3  O  ^ 


-( e-c,) 


exp • 


2  (  a%a%  f  <ye+  ay) 


2^_3, 


r  c  i  c  3  •> 

C2  exp{  -4-3-  }  =  Kl^K2 

y2^  Jo%+o%  ^  °X 


g(  *)  = 


exp 


f  -(x-nx) 

2  4 


g(*)  =  (KjA-  +  K2)p^(^) 

where  constants  and  K2  are: 

„  _  »0a%  ~ 


‘1  ~  „2  ,  2 

tf0  +  <V 


Thus,  equation  (C.l)  becomes: 


ri+  1 


2  2 

a9  ‘V 


I  -«  T  A 

x  p  x(  x)  Ax 

»v  . 


n  ~~  Ki  x~ 


a  J+ 1 

J 


+  K9, 


d  a- 


Vi  =  1 ,  •  •  • ,  N 


Expanding  equation  (3.12): 

I_«  92  ~  2  yj- 1  *<*i>  +  -^i-i  p^*i> 

=  J  92  P^A-j.-ajp^^dA-  -  2  Xj  g(^j)  +  y\  F 

=*  2  ( Xj-  -  gCA-j.)  =  (r“  -  r j_ ! ) 


Now , 


(C.5) 


(C.  6) 

( C  .  7  ) 


(  C  .  8 ) 

x  i ) 

( C  .  9  ) 
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&(  x  j)  -  l  6  P0|^(0|^j)  P  x(  Xj)  d0  =  K^.  +  K2 

K1  xi  +  K2  =  — - Vi  =  1 ,  •  •  •  N  (C.10) 

2 

And  now  equations  (C.8)  and  (C.10)  are  the  results  quoted 
in  equations  (3.26)  and  (3.25),  respectively. 

C.2.  Absolute  Error  Cost 


The  posterior  density  function  for  this  example 
satisfies  the  conditions  so  that  equation  (3.17)  may  be 
used  for  when  the  cost  function  is  the  absolute  error 
cost . 

*(*i)  =  9  Pei^el*,-)  p *(*■;)  de  =  k1a-j.  +  k2 

=»  gup  =  k1a-j.  +  k2  =  --»■  -.-i-1,  (C.ll) 

Vi  =  1 ,  • •  •  N 

And  equation  (3.19)  may  be  used  for  the  level  condition: 


,xi+ 1 

* 

x  ■ 

1 

p  &  |  x(  9  |  x)  d  0 

O 

which  from  (C.5)  implies: 


Vx(x)&x  = 


1 

P x{  x)  dx 


[*xi+ 1 

r  Vi  -  &(x) 

1 

erf 

i 

p  „(  x)  dx  =  - 

x  i 

.(a§o“/(o|+o*))-5. 

A  2 

i+1 


p  x)  dx  (C.12) 


xi 
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C . 3 .  Uniform  Error  Cost 


The  posterior  density  function  satisfies  the 
necessary  conditions  so  that  equation  (3.22)  may  be  used 
for  the  threshold  condition  and  the  uniform  error  cost 
f unct ion . 


=  S_co  9  Pei/0!*;)  dd  =  Kl*i  +  K2 

=»  g(Xj)  =  K  x  Xj  +  K2  =  LL-LJ±LLt  (C.l  3) 

Vi  =  1 ,  •  •  *N 

Likewise,  the  level  condition  of  equation  (3.24)  may  be 
used : 


P0(rJ-V2)  J  x-(  yfti/2.)  )  dx 

xi 

n^i+l 

=  P0(yi-+4/2)  P^A(W(/i+A/2))  dx 


(C. 14) 


By  evaluating  (C.14)  with  the  Gaussian  probability  density 
functions  for  the  example: 


exp  (  A(  y  j  -  M0)/2a|)  jerf 


S>  {< 


ler  f 

ki+l-(7/V2)-MJ 

L 

l 

-  erf 


xr±yf±L  2)~M/vl 


N 


*  exp  ( -&(  y j  -  *9)/2o%)  |erf^J+1  (  ^ 


-A/2)-m^1 


(C. 15) 


-  erf 


ArJ.-(ri-4/2)-M^ 


« 


i 


( 
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APPENDIX  D 


Pictures 
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APPENDIX  D 


Images 


The  following  pages  contain  sample  pictures  of  the 
quantization  of  a  noisy  image  source  with  the  Max-Lloyd, 
uniform,  and  minimum  risk  quantizers  as  described  in 
chapter  5.  The  source  is  assumed  Gaussian  with  the  a 
prion  statistics  obtained  by  sampling  a  window  placed 
about  the  face  of  the  girl  in  the  image.  These 
statistics  were:  m9  =  101.71,  and  o&  =  25.1  on  a 

256-level  intensity  gray  scale.  The  quantizers  were 
obtained  by  scaling  by  oQ  and  shifting  by  a  Max-Lloyd 

(with  squared-error  distortion  function)  quantizer 
designed  for  a^ source  ~  N(0,1).  The  is  based  on  the 
conversion  of  any  random  variable  from  a  normal  density 
function  to  a  standard  normal  density  through: 


s 


x  ~  N( M, s) , 
z  ~  N(0, 1) 


Note  how  at  the  lower  SNR  rates,  the  minimum-risk 
quantizer  levels  converge  to  the  mean  of  the  source. 


This  is  the  reason  for  the  "washed-out"  or  low  contrast 
appearance.  Also  note  how  the  noise  mean  adversely 
affects  all  of  the  quantizers  with  the  exception  of  the 
minimum  risk.  While  this  may  not  be  the  ideal 
demonstration  of  the  quantization,  the  results  do 
indicate  the  degree  of  success  of  quantizing  noisy 
sources  relative  to  other  approaches. 
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/* - Quantizer. h - ♦/ 

♦define  QMAX  128 

♦define  LEVEL(x.n)  ♦(x->level+n) 

♦define  THRESH(x,n)  *(x->thresh+n) 

♦define  NLEVELS(x)  x- >num_levela 

typedef  struct  quantizer  { 

int  num_ levels; 
double  ♦level; 
double  ♦thresh; 

}  QUANT,  ♦QUANTPTR; 

/♦ - Function,  h - ♦/ 

♦define  INTERVALS  5000 
♦define  INFINITY  10. 

♦define  FMAX  10 

♦define  FPAHM(x)  x-> parameter 

♦define  FDPAHM(x,n)  (x->parameter+n)->dval 

♦define  FFPAHM(x.n)  (x->parameter+n)->function 

♦define  FFUNCT(x.n)  (x->parameter+n)->fnct 

♦define  FNUM(x)  x— >num_parameters 

struct  funct  { 

union  parm  { 

double  dval; 

double  (♦function)  (); 

struct  funct  ♦fnct; 

}  ♦parameter; 
int  num  parameters; 

}; 


typedef  union  parm  PARM,  ♦PARMPTH; 
typedef  struct  funct  FUNCT,  ♦FUNCTPTR; 
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/* - Function,  c - */ 

♦include  <stdio.h> 

♦include  <stdlib.h> 

♦include  <function.h> 

FUNCTPTR  f_alloc(n) 
int  n; 

{ 

FUNCTPTR  f; 

f  =  (FUNCTPTR)  calloc( 1 , sizeof (FUNCT) ) ; 

FNUM(f)  =  n; 

FPAHM(f)  =  (PARMPTR)  cal loc(FMAX, sizeof (FARM) ) ; 
return(f) ; 

} 


int  free_funct(f ) 
FUNCTPTR  f ; 

{ 

extern  int  free(); 
free(FPARM(f) ) ; 
free(f) ; 

} 
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/% -  Quantizer. c  (Newquant)  - */ 

♦include  <stdio.h> 

♦include  <stdlib.h> 

♦include  <quantizer.h> 

QUANTPTR  q_alloc(n) 
int  n; 

{ 

QUANTPTR  quant; 

quant  =  (QUANTPTR)  calloc(l,aizeof (QUANT) ) ; 

NLBVELS( quant)  =  n; 

quant->level  =  (double  *)  calloc(QMAX, aizeof (double) ) ; 
quant-> thresh  =  (double  *)  calloc(QMAX, aizeof ( double) ) ; 
return (quant) ; 

} 


int  zero_quant(q) 

QUANTPTR  q; 

( 

int  i; 

for( i  =  0;  i  <=  NLEVELS(q) ;  ++i)  { 
THRESH(q.i)  =  0.; 

LEVEL(q.i)  =  0.; 

} 

} 


int  print_quantizer(q) 

QUANTPTR  q; 

{ 

int  i,n; 

n  =  NLEVELS(q) ; 
for(i=0;  i<n;  ++i) 

printf  ("Xlf  XlAn",  THRESH(q,  i) ,  LEVEL(q,  i) ) ; 

} 
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int  fprint_quantizer(q,fp) 

QUANTPTR  q; 

FILE  *fp; 

{ 

int  i,n; 
n  =  NLEVBLS(q); 
for(i=0;  i<n;  ++i) 

fprintf (fp, "*lf  *lf\n", THRESH(q, i) , LEVEL(q, i) ) ; 

} 


int  free_quant(q) 
QUANTPTR  q; 

{ 

extern  int  free( ) ; 

free(q->thresh) ; 
free(q-> level) ; 
free(q) ; 
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/* - 

Function  to  quantize  a  .value  with  a  given  quantizer. 

Quantizer  is  assumed  to  have  the  form: 


xl 

yl  : 

where  level  yj  corresponds  to 

x2 

y2  : 

an  x  in  the  region  (xj,xj+l). 

xn 

yn  i 

also  xn+1  is  assumed  -^-infinity, 

and  no  value  is  stored  for  it. 


•*/ 


double  quantize(value,q) 
double  value; 

QUANTPTR  q; 

{ 

int  ptr  =  0; 

while  (  (value  >  THRESH(q.ptr) )  &&  (ptr  <  NLEVELS(q) )  ) 
ptr++; 
ptr — ; 

return(  LEVEL(q.ptr)  ); 

} 


/* - 

-  END  of  Quantizer. c  — 

- */ 

/* - 

Integral. c  - 

- */ 

♦include  <stdio.h> 
♦include  <stdlib.h> 
♦include  <math.h> 
♦include  <function.h> 


double  dabs(x) 
double  x; 

{ 

if  (  x  <  0. ) 

return (  -x  ); 
else 
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} 


return (  x  ) ; 


double  integral(xl,xu,f) 
double  xl; 
double  xu; 

FUNCTPTH  f; 

{ 

extern  double  intdiv( ) , dabs( ) ; 
double  areal, area, diff, tmp.eps  =  1.; 
int  n; 

if  (xl  >  xu)  { 

tmp  =  xl;  /*  Want  to  integral  from  lower  value  */ 

xl  =  xu;  /#  to  the  higher  value.  */ 

xu  =  tmp; 

} 

areal  =  intdiv(xl,xu, f , 1) ; 
n  =  2; 

area  =  intdiv(xl,xu, f ,n) ; 
while(  ( dabs ( area)  >  0.0001) 

/*  &&  (dabs (diff  =  area  -  areal)  >  0.0001) 

*/ 

&&  ((daba((diff  =  area  -  areal)  /  area)  *  100.)  >  eps)  )  { 
areal  =  area; 
n++; 

area  =  intdiv(xl,xu,f,n) ; 

/*  printf("n:Xd  area: *lf\n",n, area); 

*/ 

} 

return(  area  ); 


double  intdiv(xl,xu,f,n) 
double  xl; 
double  xu; 

FUNCTPTR  f; 
int  n; 

{ 

double  intgrl(); 

double  lower , upper, incr, area  =  0.; 
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int  i; 

incr  =  (xu  -  xl)  /  n; 
lower  =  xl; 

for(i  =  0;  i  <  n;  ++i)  { 
upper  =  lower  +  incr; 
area  +=  intgrl(lower,upper,f); 
lower  =  upper; 

} 

return (  area  ) ; 

} 


double  intgrl(xl,xu, f) 
double  xl; 
double  xu; 

FUNCTPTH  f; 

{ 

double  (*fct)  (); 
double  a^.xp.xm.c.y; 

fct  =  FFPARM(f.O); 
a  =  0.5B+0  *  (xu+xl); 
b  =  xu-xl; 

c  =  0. 49863 193092474078E+0  *  b; 
FDPAHM(f.l)  =  a  +  c; 
xp  =  (*fct)  (f); 

FDPAHM(f.l)  =  a  -  c; 
x*  =  (*fct)  (f); 

y  =  0.35093050047350483E-2  *  (xp  +  xm); 
c  =  0. 492805755772634 17E+0  *  b; 
FDPAHM(f.l)  =  a  +  c; 
xp  =  (*fct)  (f); 

FDPAHM(f.l)  =  a  -  c; 
xm  =  (*fct)  (f); 

y  =  y+0. 8137197365452835E-2  *  (xp  +  xm) 
c  =  0 . 48238112779375322E+0  *  b; 
FDPAfiM(f.l)  =  a  +  c; 
xp  =  (*fct)  (f); 

FDPAfiM(f.l)  =  a  -  c; 
xm  =  (*fct)  (f); 

y  =  y+0. 1269603265463 1030E-1  *  (xp  +  xm 
c  =  0 . 467 45303796886984E+0  *  b; 
FDPAHM(f.l)  =  a  +  c; 
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xp  =  (*fct)  (f); 

FDPAHM(f.l)  =  a  -  c; 
xa  a  (*fct)  (f); 

y  =  y+0. 17136931456510717E-1  *  (Xp  +  xm); 
c  =  0 . 44816057788302606E+0  *  b; 
FDPAHM(f.l)  =  a  +  c; 
xp  =  (*fct)  (f); 

FDPAHM(f.l)  =  a  -  c; 
xa  =  (*fct)  (f); 

y  =  y+0. 21417949011113340E-1  *  (xp  +  xm); 
c  =  0.42468380686628499E+0  *  b; 
FDPAHM(f.l)  =  a  +  c; 
xp  =  (*fct)  (f); 

FDPARM(f.l)  =  a  -  c; 
xa  =  (*fct)  (f); 

y  =  y+0 - 2549902963 1 188088E- 1  *  (xp  +  xa); 
c  =  0. 39724189798397 120E+0  *  b; 
FDPAEM(f.l)  =  a  +  c; 
xp  =  (*fct)  (f); 

FDPARM(f.l)  =  a  -  c; 
xa  =  (*fct)  (f); 

y  =  y+0 . 29342046739267774B-1  *  (xp  +  xm); 
c  =  0.366091059370144848+0  *  b; 
FDPAHM(f.l)  =  a  +  c; 
xp  =  (*fct)  (f); 

FDPAHM(f , 1)  =  a  -  c; 
xa  =  (*fct)  (f); 

y  =  y+0 . 32911111388180923E-1  *  (xp  +  xa) ; 
c  =  0. 331522133465 10760B+0  *  b; 

FDPAHM(f.l)  =  a  +  c; 
xp  =  (*fct)  (f); 

FDPAHM(f.l)  =  a  -  c; 
xm  =  (*fct)  (f) ; 

y  =  y+0.36172897054424253E-l  *  (xp  +  xm) ; 
c  =  0. 293857878620381 16E+0  *  b; 

FDPAHM(f.l)  =  a  +  c; 
xp  =  (*fct)  (f); 

FDPAHM(f , 1)  =  a  -  c; 
xa  =  (*fct)  (f); 

y  =  y+0. 39096947893535153E-1  *  (xp  +  xm); 
c  =  0. 253449954466 11470B+0  *  b; 

FDPAHM(f.l)  =  a  +  c; 
xp  =  (*fct)  (f); 
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FDPAHM(f.l)  =  a  -  c; 
xm  =  (*fct)  (f); 

y  =  y+0. 41655962 113473378E-1  *  (xp  +  xm); 
c  =  0.2 106756380653 1767E+0  *  b; 

FDPAHM(f , 1)  =  a  +  c; 
xp  =  (*fct)  (f); 

FDPAHM(f.l)  =  a  -  c; 
xm  =  (*fct)  (f); 

y  =  y+0.43826046502201906E-l  *  (xp  +  xm) ; 
c  =  0. 16593430 114 106382E+0  *  b; 

FDPAHM(f , 1)  =  a  +  c; 
xp  =  (*fct)  (f); 

FDPAHM(f.l)  =  a  -  c; 
xm  =  (*fct)  (f); 

y  =  y+0 . 45586939347881942E-1  *  (Xp  +  xm) ; 
c  =  0.1 19643681 12606854E+0  *  b; 

FDPAHM(f.l)  =  a  +  c; 
xp  =  (*fct)  (f); 

FDPAHM(f.l)  =  a  -  c; 
xm  =  (*fct)  (f) ; 

y  =  y+0 . 46922199540402283E-1  *  (xp  +  xm); 
c  =  0 . 7223598079139825E-1  *  b; 

FDPAHM(f.l)  =  a  +  c; 
xp  =  (*fct)  (f); 

FDPABM(f.l)  =  a  -  c; 
xm  =  (*fct)  (f) ; 

y  =  y+0 . 47819360039637430E-1  *  (xp  +  xm); 
c  =  0.24 153832843869 158E-1  *  b; 

FDPAHM(f.l)  =  a  +  c; 
xp  =  (*fct)  (f); 

FDPAHM(f.l)  =  a  -  c; 
xm  =  (*fct)  (f); 

y  =  b  *  ( y+0 . 48270044257363900E-1  *  (xp  +  xm)); 
return  (y); 


l 


-112- 


Minimax-Risk  Program 

This  program  is  intended  to  determine  the  minimax  risk 
quantizer  for  a  known  Gaussian  p.d.f.  It  uses  determines 
mines  the  Lagrange  multipliers  needed  for  the  maximization 
of  the  I (noise)  function,  such  that  that  maximum  is  zero 
and  the  Lagrange  multipliers  are  greather  than  or  equal 
to  zero. 


♦include  <stdio.h> 
♦include  <stdlib.h> 
♦include  <string.h> 
♦include  <math.h> 


♦define  INFINITY  30000.0 


Variable  description: 


C  -  Cost  function 

qthr,mthr  -  quantizer  threshold  set 

qlev.mlev  -  quantizer  level  set 

tl,tu  -  threshold  lower  and  upper  limits 

nl,nstep,nu  -  noise  limits  and  increment 

d2min  -  fidelity  criterion  for  lambda2  =  .00001 

d2max  -  maximum  expected  lambda2 

120  -  minimum  expected  lambda2  =  0 

constraint  -  on  mean  square  power  of  the  noise 

mmrisk  -  minimax  risk 

limit  -  bound  on  the  distortion  measure 
lambdal,  lambda2  -  Lagrange  multipliers 
lower, step, upper  -  limits  and  step  on  theta 
mean,  std  -  on  theta 


♦define  C(theta,y)  (theta  -  y)  *  (theta  -  y) 
♦define  EVER  ( ; ; ) 

♦define  N  4 


♦define  max(a,b) 
♦define  min(a,b) 
♦define  abs(a) 

FILE  *out; 


(((a)  >  (b))  ?  (a)  :  (b)) 
(((a)  <  (b))  ?  (a)  :  (b)) 
(((a)  <  0.0)  ?  -(a)  :  (a)) 


float  qthr(5],  qlev[4]; 
float  mthr[5] ,  mlev[4] ; 
float  tl, tu,mdl,md2,md3; 
float  nl,nstep,nu; 
float  mean,  std,  twostd; 
float  d2min,  d2max,  120; 
float  constraint,  limit; 
float  lambdal,  lambda2; 
float  mmrisk; 
float  lower, step, upper; 


/*  limits  on  noise  */ 


/♦  Optimum  minimax  risk  */ 


/* -  Gaussian  p.d.f.  of  N( mean, std)  - */ 


float  p_normal(x) 
float  x; 

{ 

extern  float  mean,  std; 
extern  double  exp(); 
float  arg; 

arg  =  -(x-mean)*(x-mean)/(2. *std*std) ; 

if  (arg  <  -30.0) 
return (0. 0) ; 
else 

return(  0.3989422804  *  exp( (double)  arg)  /  std); 
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int  print  quantizer() 

{ 

extern  float  qthr[],  qlev[]; 
int  i; 

for(i  =0;  i  <  N;  i++) 

printf("thr:  Xe  lev:  Xe\n",qthr[i] ,qlev[i] ) ; 
printf ( "thr:  Xe\n" , qthr[N] ) ; 

} 


int  fprint_quantizer() 

{ 

extern  FILE  *out; 

extern  float  d2min,d2max, 120, constraint,mmrisk; 

extern  float  atd, limit; 

extern  float  qthr(],  qlev[]; 

extern  float  tl,atep,tu,nl,nstep,nu; 

int  i; 

fprintf ( out , "std:  Xe\n",std); 

fprintf (out, "d2min:  Xe  d2max:  Xe\n",d2min,d2max) ; 
fprintf (out, "120:  Xe\n",120); 
fprintf (out, "tu:  Xe  nu:  Xe\n", tu.nu) ; 
fprintf ( out , "constraint :  Xe\n" , constraint ) ; 
fprintf (out, "limit:  Xe\n", limit) ; 

for(i  =0;  i  <  N;  i++) 

fprintf (out, "thr:  Xe  lev:  Xe\n",qthr[i] ,qlev(i] ) ; 
fprintf (out, "thr:  Xe\n" , qthr[N] ) ; 
fprintf ( out , "mmrisk :  Xe\n" , mmr isk ) ; 

} 
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/* —  Writes  out  an  intermediate  quant,  as  it  finds  them  — */ 


int  ffprint_quantizer( ) 

{ 

extern  FILE  *fopen( ) ; 
extern  float  marisk,d2min,d2max, 120; 
extern  float  constraint, std, limit; 
extern  float  qthr[] »  qlev[]; 
extern  float  tu,  nu; 

FILE  ^output; 
char  outname[50]; 
int  i; 

strcpy(outname,"mm40out  XXXXXXXX  b"); 

mktemp(outname) ; 

output  =  fopen ( outname , "w" ) ; 

fprintf( output, "tu:  Xe  nu:  Xe\n",tu,nu) ; 

fprintf ( output , "d2min : Xe  d2max : Xe\n" , d2min , d2max ) ; 

fprintf (output, ”120:  Xe\n",120) ; 

fprintf (output, "std:  Xe\n", std) ; 

fprintf (output, "constraint:  Xe\n", constraint) ; 

fprintf (output, "limit:  Xe\n", limit) ; 

for(i  =0;  i  <  N;  i++) 

fprintf (output, "thr:  Xe  lev:  Xe\n",qthr[iJ ,qlev[i] ) ; 
fprintf (output, "thr:  Xe\n",qthr[N) ) ; 

fprintf (output, "mmrisk:  Xe\n",mmrisk) ; 
fc lose (output) ; 

} 
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/* -  Stores  the  current  quantizer  as  minimax 


int  copy  quantizerQ 

{ 

extern  float  qthr[  ]  .qlevQ  ,mthr{]  ,alev[  ] ; 
int  i; 

for(i  =0;  i  <  N;  i++)  { 

■thr( i]  =  qthr[ i] ; 
mlev[i]  =  qlev[i]; 

} 

} 


/* - -  Quantizes  a  s ample 


float  q(x) 
float  x; 

{ 

extern  float  qthr(] ,  qlev[]; 
register  int  k  =  0; 

while(  (k  <  (N  -  1))  &&  (qthr[k+l]  <=  x)  ) 
k++; 

if (qthr[N]  <=s  x) 

return (qlev(N-l] ) ; 
else 


return (qlevfk] ) ; 


/* -  Finds  maximum  of  11(12)  over  the  noise  range  — 


- */ 


float  xxeval( noise) 
float  noise; 

{ 

extern  float  p_normal(),  lambda2,  mean,  limit; 
extern  float  qthrf],  qlev[],  lower,  step,  upper; 
float  i,  theta; 
float  x,  q( ) ; 
float  sum,val; 

sum  =  0.0; 

for( theta  =  lower;  theta  <=  upper;  theta  +=  step  )  { 

x  =  theta  +  noise; 
if((val  =  C(theta,q(x)))  >  limit) 
sum  +=  limit  *  p_normal ( theta) ; 
else 

sum  +=  val  *  p_normal ( theta ) ; 

/* 

printf(”n:*e  t:Xe  val :Xe\n", noise, theta, val ) ; 

*/ 

> 

/* 

printf ( "sum: *e\n" , sum) ; 

*/ 

sum  *=  step;  /*  Riemann  sum  *  step  */ 

sum  -=  lambda2  *  noise  *  noise; 

/* 

printf ("noise:  Xe  final  sum: \e\n”, noise, sum) ; 

*/ 

return (sum) ; 

} 
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/* -  Determines  lambdal(lambda2)  - */ 


float  11(12) 
float  12; 

{ 

extern  float  lambdal,  lambda2,  mean; 
extern  float  nl,nstep,nu; 
float  noise; 
float  xxeval(),  val; 

lambdal  =  -INFINITY; 
lambda2  =  12; 

for (noise  =  nl;  noise  <=  nu;  noise  +=  nstep)  { 
val  =  xxeval(noise) ; 

lambdal  =  (val  >  lambdal)  ?  val  :  lambdal; 

/* 

pr intf ( "noise : Xe  val : Xe\n" .noise , val ) ; 

*/ 

} 

return ( lambdal) ; 

} 


/* -  Performs  5-point  minimization  over  lambda2  - */ 

/# -  as  outlined  in  Bath's  dissertation.  - */ 


float  12 () 

{ 

extern  float  constraint,  lambdal,  120,  d2max,  d2min; 
float  j(4],  12,  d2,  12star,  11 ( ) ; 
int  k; 

12star  =  120; 
d2  =  d2max; 

while(d2  >  d2min)  { 

for(k  =0;  k  <  4;  k++)  { 

12  =  12star  +  k  *  d2; 

j[k]  =  11(12)  +  12  *  constraint; 

/* 
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printf ("12star:  Xe  12:  Xe  j[Xd):  Xe  d2:  Xe\n", 

12a tar, 12,k, j(k] ,d2) ; 

*/ 

} 

if  (J[0]  <=  j[l] ) 
d2  /=  4.0; 

if((j[0]  >  j[l] )  &&  (j[l]  <=  J[2] )) 
d2  /-  2.0; 

if((j(0]  >  j[l])  &&  (j(l]  >  j[2] )  &&  (j[2]  <=  jf3] ))  { 
12a tar  +=  d2; 
d2  /=  2.0; 

} 

if((j[0]  >  J[l] )  &&  (j[l]  >  j [2] )  &&  ( j[2]  >  j [3] ) )  { 
12atar  +=  2.0  *  d2; 

d2  /-  2.0; 

> 

} 


return( ll(12star)  +  12star  ♦  constraint); 


Determines  the  minimax  risk  quantizer 


/*■ 


*/ 


float  riskq() 

{ 

extern  FILE  #out; 

extern  float  tl, nstep,  tu; 

extern  float  mdl ,  md 2,  md3; 

extern  float  qthr[],  qlev[]; 

extern  float  constraint,  mean; 

extern  float  mmrisk; 

extern  float  lambdal,  lambda2; 

int  copy_quantizer( ) ,  fprint_quantizer( ) ,  print_quantizer( ) ; 

int  ffprint_quantizer( ) ; 

float  end,  tlpl,  tlp2; 

int  flthr,  fllev; 

float  12(); 

float  curr_risk,  last; 

mmrisk  =  INFINITY; 

tlpl  =  tl  +  nstep; 
tlp2  s  tl  +  2.  *  nstep; 

for(qthr[l]  =  md2;  qthrfl]  >  tlp2;  qthr[l]  -=  nstep)  { 
for(qlev[0]  =  md3;  qlev[0]  >  tlpl;  qlev[0]  -=  nstep)  { 
fllev  =  0; 

last  =  INFINITY;  /*  restart  risk  calc  for  levO —  */ 
end  =  qlev(0] ; 

for(qlev[l]  =  mdl;  qlevfl]  >  end;  qlevfl]  -=  nstep)  { 


/* 

*/ 


qthr[3]  =  qthr[2j  +  (qthr[2]  -  qthr[l]); 

qlev[2]  =  qthr[2]  +  (qthr[2]  -  qlev[l]); 

qlev[3]  =  qthr[2]  +  (qthr[2]  -  qlevfO]); 

print_quantizer( ) ; 

curr_risk  =  12(); 


if(curr_risk  >  last)  { 
qlev[l]  =  end; 

} 
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last  =  curr_ri.sk; 


if(curr_risk  <  =  mmrisk)  { 

flthr  =1;  /*  mmrisk  for  this  thr  */ 

copy_quantizer( ) ; 

md3  =  qlev[0];  /*  new  start  for  outer  lev*/ 

fprint_quantizer( ) ; 
mmrisk  =  curr_risk; 
printf ("mnrisk:  \le\n" ,  mmrisk); 
print_quantizer( ) ; 

fprintf( out, "mmrisk:  Xe\n" .mmrisk) ; 

ffprint_quantizer( ) ; 


} 

} 

> 

} 


/* - Evaluates  the  risk  for  a  specific  quantizer - */ 


float  calc_risk() 

{ 

extern  float  constraint; 
extern  float  qthr[],  qlev[]; 
extern  float  mmrisk; 
extern  float  mean; 

extern  int  fprint_quantizer( ) ,  print_quantizer( ) ; 
float  q(),x; 
float  12(); 
int  i; 

for  EVER  { 
mmrisk  =  INFINITY; 

for(i  =0;  i  <  2;  i++)  { 

printf("Enter  thr[*d] ,  level[Xd]:  ",i,i); 
scanf("Xe  Xe",&qthr[i] ,&qlev[i] ) ; 

} 
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qthr[2] 
qthr[3] 
qthr[4] 
qlev[2] 
qlev [ 3 ] 


mean; 

qthr[2]  +  (qthr[2] 
qthr[2]  +  (qthr[2] 
qthr[2]  +  (qthr[2j 
qthr{2]  +  (qthr[2] 


-  qthr[l] ) ; 

-  qthr(OI); 

-  qlev[l] ) ; 

-  qlevfOJ); 


mmrisk  =  12 ( ) ; 

print_quantizer( ) ; 

printf ("mmrisk:  *e\n", mmrisk) ; 


return (mmrisk) ; 


/* - Main  driver  program 


main( ) 

{ 

extern  FILE  *out; 

extern  float  tl,  tu,  mthr[],  mlev[J; 

extern  float  nl,  nstep,  nu; 

extern  float  constraint,  mean,  std,  twostd; 

extern  float  limit; 

extern  float  d2min,  d2max,  120; 

extern  float  lower,  step,  upper,  mdl,  md2,  md3; 

extern  double  sqrt(); 

char  oname[50] ; 

int  fprint_quantizer( ) ,  print_quantizer( ) , i; 
int  ffprint_quantizer( ) ; 
float  calc_risk(),  riskq(); 

printf ("Minimax-Risk  quantizer  program. \n\n" ) ; 

printf("Enter  output  filename:"); 
gets(oname) ; 

out  =  fopen(oname, "w") ; 


•*/ 
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printf ("Enter  tu  (mean  will  be  added):  "); 
scanf ( "Xe" , &tu) ; 

printf ("Enter  source  mean  and  std:"); 
scanf ("Xe  Xe",&mean,&std) ; 

tl  =  mean  -  tu; 
tu  =  mean  +  tu; 

twostd  =  2.0  *  std; 

lower  =  mean  -  twostd; 
upper  =  mean  +  twostd; 
step  =  (upper  -  lower)  /  25.; 

printf("tl:Xe  tu:Xe  2std:Xe  lower : Xe  step:Xe  upper : Xe\n"  , 
tl.step, tu, twostd, lower, step, upper) ; 

printf ("Enter  constraint:  "); 

scanf ( "Xe" , &constraint ) ; 

printf ( "constraint : Xe\n" , constraint ) ; 

nu  =  mean  +  2.0  *  sqrt( (double)  constraint); 
nl  =  mean; 

nstep  =  (nu  -  nl)  /  25.; 

printf ("nl:Xe  nstep: Xe  nu: \n",nl, nstep, nu) ; 

printf ("Enter  bound  on  cost:  "); 

scanf ("Xe" ,&limit) ; 

printf (" limit :Xe\n", limit) ; 

printf ("Enter  d2min,  d2max:  ”); 
scanf ("Xe  Xe”,&d2min,&d2max) ; 

printf ("Enter  120:  "); 
scanf ( "Xe" , &120 ) ; 

printf ("Starting  quant izer\n" ) ; 

printf ("Enter  Levell  Thresh  Level2  (w.r.t.  mean):") 
scanf ("Xe  Xe  Xe",&mdl,&md2,&md3) ; 
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/*  Assume  a  symmetric  quantizer  */ 
qthrfO]  =  tlj 
qthr[2]  =  mean; 
qthr[4]  =  tu; 

mdl  =  qthr [2]  -  mdl; 
md2  =  qthr [2]  -  md2; 
md3  =  qthr[2]  -  md3; 

printf ( "d2ain:  Xe  d2max:  Xe\n" , d2min, d2max) ; 
printf('*120:  Xe  \n",120); 
printf("limit:  Xe\n" , limit) ; 

fprintf(out,  ’’limit:  Xe\n", limit)  ; 

fprintf (out, "d2min:  Xe  d2max:  Xe\n”,d2min,d2max) ; 

fprintf (out, "120:  Xe\n",120); 

/*  Replace  with  calc_risk  for  a  specific  quantizer  testing  */ 
riskq(); 

printf ("Minimax-risk:  Xe  c: Xe\n" , mmrisk , constraint ) ; 
for(i  =0;  i  <  N;  i++) 

printf ("thr:  Xe  lev:  Xe\n",mthr[i] ,mlev[i] ) ; 
fprintf (out, "Minimax-risk:  Xe  c: Xe\n",mmrisk, constraint) ; 
for(i  =0;  i  <  N;  i++) 

fprintf(out, "thr:  Xe  lev:  Xe\n",mthr[i] ,mlev[i] ) ; 

fclose(out) ; 
exit(0) ; 
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RABS11.C 

Program  to  determine  the  minimum  risk  quantizer  for  the 
absolute-error  cost  functions  and  Gaussian  source  +  Gaussian 
noise. 

- */ 


♦include  <stdio.h> 

♦include  <stdlib.h> 

♦include  <math.h> 

♦include  <function.h> 

♦include  <quantizer.h> 

/* - 

Variable  descriptions: 
q  -  quantizer 

n_mean,  n_std  -  noise  statistics 

xmean,  x_std  -  source  statistics 

r_mean,  r_std  -  observation  statistics 

n_inf ,x_inf , r_inf  -  "infinities  for  resp.  densities 

- */ 

♦define  INCREMENT  .1 
♦define  FIDELITY  .00001 

QUANTPTR  q; 

double  increment  =  INCREMENT; 

double  n_aean,n_std,x_mean,x_std,r_mean, r_std; 
double  x_inf ,o_inf, r_inf; 
double  kl,k2,x_var,n_var,r_var; 
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double  sgn(x) 
double  x; 

{ 

if(x  <  0.) 

return  (-1.); 
else  if (x  ==  0. ) 

return  ( 0 . ) ; 
else 

return  ( 1 . ) ; 


/* -  Gaussian  p.d.f.  for  N(mean,std) 


■*/ 


double  p_nonaal(nean,std,x) 
double  mean; 
double  std; 
double  x; 

{ 

extern  double  exp(); 
double  arg; 

if  ((x  >  (5.0  *  std  +  mean))  ! I  (x  <  (-5.0  *  std  +  mean))) 
return(0. ) ; 
else  { 

arg  =  -(x-mean)*(x-mean)/(2.*std*std) ; 
return(  0.3989422804  *  exp(arg)  /  std); 

} 

} 


/* -  Integrable  p.d.f  function  - */ 

double  pnorm(f) 

FUNCTPTR  f; 

{ 

extern  double  p_normal(); 
double  arg; 

arg  =  FDPARM(f.l); 

return (  p_normal(0. , 1. ,arg)  ); 


} 
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double  p_noise(n) 
double  n; 

{ 

extern  double  n_mean,  n_std,  p_normal(); 
return (p_nonaal(n_nean, n_std,n) ) ; 

} 


double  p_x(x) 
double  x; 

{ 

extern  double  x_mean,  x_std,  p_normal(); 
return(p_noroal(x_mean,x_std,x) ) ; 

} 


/* - 

Observation  probability  density  function. 
Defined  as  the  convolution  of  p(x)  and  p(n)  at 
the  observation  point  r. 


double  p_r(f) 

FUNCTPTR  f; 

{ 

extern  double  p_nonnal(),  r_mean,  r_std; 
double  r; 

r  =  FDPARM(f,l); 

return ( p_normal ( r_mean , r_std, r) ) ; 

} 

/* -  Initialize  quantizer  - 


int  sset_initial(q, guess) 
QUANTPTR  q; 
double  guess; 

{ 

extern  double  rjnean,  r_inf; 


THRESH(q.O)  =  101.71; 


LEVEL(q.O)  =  102.0301631; 
THHESH(q.l)  =  103.0981765; 
LBVEL(q, 1)  =  102.7780127; 


} 


int  set_initial(q, guess) 
QUANTPTH  q; 
double  guess; 

{ 

extern  double  r_mean,  r_inf; 

THRESH(q,0)  =  r_mean; 
LEVEL(q,0)  =  guess; 
THRESH(q,l)  =  r_mean  +  r_std; 


} 


double  db(x) 
double  x; 

{ 

extern  double  logl0(); 
return(10.  *  loglO(x)); 

} 


double  dbinv(x) 
double  x; 

{ 

extern  double  pow(); 

retum(  pow(10.,  0.1  *  x)  ); 

} 
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-  Detetermine  if  quantizer  criteria  are  met 


/* 


■*/ 


double  abaci (rj,yj, rjpl) 
double  rj; 
double  yj; 
double  rjpl; 

{ 

extern  FUNCTPTH  f_alloc( ) ; 

extern  double  abacla( ) , p_r( ) , cum_norm( ) , intgrl( ) ; 
extern  double  kl,k2,r_mean,r_std; 

FUNCTPTH  f; 
double  a,b,xl,xu; 
f  =  f_alloc(10); 

FFPAHM(f,0)  =  abscla; 

FDPABM(f,2)  =  yj; 
a  =  intgrl ( r j , rjpl , f ) ; 
free_funct(f ) ; 

xl  =  (rj  -  r_mean)  /  r_std;  /*  find  prob  of  p(r)  between  */ 

xu  =  (rjpl  -  rjnean)  /  r_atd;  /*  rj  and  rj+1  (gauasian  so  */ 

b  =  (cum_norm(xu)-cum_nona(xl) )/2. ;  /*  can  use  F(x)  instead  of  */ 

/* 

printf(”a:*lf  b:*lf  rj:*lf  yj:»lf  rjpl:*lf\n,\a,b,rj,yj,rjpl) ; 

*/ 

return (a  -  b);  /*  general  integration.  */ 

} 


double  abscla(f) 

FUNCTPTH  f; 

{ 

extern  FUNCTPTH  f_alloc(); 
extern  double  p_r( ) ,cua_norm( ) ,sqrt( ) ; 
extern  double  kl,k2,n_var; 
extern  int  free_funct( ) ; 

FUNCTPTH  fp; 

double  r,  yj,  pr_r,  mean,  std; 

fp  =  f_alloc(10); 
r  =  FDPAEW(f.l); 

FDPAHM(fp.l)  =  r; 
yj  =  FDPAHM( f , 2) ; 
pr_r  =  p_r(fp); 
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free_funct(fp); 

mean  =  kl  *  r  +  k2; 

atd  =  aqrt(  n_var  *  kl); 

return(  cum_norm(  (yj-mean)/std  )  *  pr_r  ); 


double  cum_norm(x) 
double  x; 

{ 

extern  double  pnorm(),  intgrlQ; 
extern  FUNCTPTR  f_alloc(); 
extern  int  f ree_funct ( ) ; 

FUNCTPTR  f; 
double  y; 

f  =  f_alloc( 10) ; 

FFPAHM(f , 0)  =  pnona; 
y  =  intgrl(-5.0,x,f); 
free_funct(f) ; 
return (y) ; 


double  absc2 ( y jml , r j , y j ) 
double  yjml.rj.yj; 

{ 

extern  FUNCTPTR  f_alloc(); 
extern  int  free_funct( ) ; 

extern  double  intgrl( ) ,  cum_norni( ) ,  absc2a( ) ; 
extern  double  kl,k2,n_var; 

FUNCTPTR  fl; 
double  a,b,mean,std; 


fl  =  f_alloc(10); 

FFPARM(fl.O)  =  absc2a; 
FDPARM(fl,2)  =  rj; 
a  =  intgrl(yjml,yj,fl); 
mean  =  kl  *  rj  +  k2; 
std  =  sqrt(  kl  *  n_var  ); 
b  =  (yjml-yj)/2. 

+  yj*cum_norm( (yj-mean)/std) 


-  yjml*cum_norm( (yjml-mean)/std) 


free_funct(fl) ; 


/* 

printf ("a:\lf  b:*lf  yjml:*lf  rj:*lf  yj:*lAn\a,b,yj,nlf  rj,yj> 
return(a  -  b); 

} 


double  absc2a(f) 

FUNCTPTR  f; 

{ 

extern  double  p_normal( ) ,kl, k2,n_var, sqrt() ; 
double  x,y,rj, mean, std; 
x  =  FDPAHM(f.l); 
rj  =  FDPAHM(f,2); 

■ean  =  kl  *  rj  +  k2; 
std  -  sqrt(n_var  *  kl); 
y  =  x  *  pnornal (mean, std, x); 
retum(y); 

} 


double  condition(f) 

FUNCT*  fR  f; 

{ 

extern  double  fabs( ) ,agn( ) ; 
extern  double  r_std,r_»ean,r_inf; 
double  (*c)(); 

double  xl,x2,x3,diff, laat_diff , add; 

c  =  FFPAHM(f.O); 
xl  =  FDPARM(f , 1) ; 
x2  =  FDPAHM(f, 2) ; 
x3  =  FDPAHM(f,3); 

if(  x3  ==  0.  )  {  /*  first  time  calc  */ 

x3  =  xl  +  r_inf;  /*  gue9S  */ 

add  =  INCREMENT; 
last_diff  =  (*c)  (xl,x2,x3) ; 

/* 

printf ("xl: SJlf  x2:*lf  x3:%lf  diff  :Xlf\n",xl,x2,x3,  last_diff) 
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We’ve  bracketed  x3  now  and  only  monitor  sign  change  to 
determine  direction  of  add...  It  will  continuously  be 
divided  in  half  since  we  know  both  bracket  limits. 


while  (  (fabs(diff)  >=  FIDELITY)  &&  (faba(add)  >=  FIDELITY)  )  { 
add  /=  (sgn(diff)  !=  agn(laat_diff))  ?  -2.  :  2.; 
laet_diff  =  diff; 
x3  +=  add; 

diff  =  (*c)  (xl,x2,x3) ; 

/* 

printf("xl:*lf  x2:*lf  x3:*lf  diff :*lf\n’\xl,x2 ,x3, diff) ; 

*/ 

} 

return(x3); 

} 


double  find_quant(q) 

QUANTPTH(q) ; 

{ 

extern  double  abscl(),  absc2(),  condition(),  r_mean,  r_inf; 
extern  FUNCTPTR  f_alloc(); 

FUNCTPTR  f; 

extern  int  free_funct{); 
double  thr.lev; 
int  i,nlev; 

f  =  f_alloc(10) ; 
nlev  =  NLEVELS(q); 

for(i  =  0;  i  <  (nlev  -  1);  ++i)  { 

FFPAHM(f.O)  =  abaci; 

FDPABM(f.l)  =  THRESH(q.i); 

FDPAHM(f ,2)  =  LEVEL(q, i) ; 

FDPAHM(f ,3)  =  THRESH(q, i+1) ; 

THHESH(q, i+1)  =  condition( f ) ; 

FFPAHM(f.O)  =  abac2; 

FDPAfiM(f.l)  =  LEVEL(q,i); 

FDPAHM(f ,2)  =  THRESH(q, i+1); 

FDPABM(f , 3)  =  LEVEL(q, i+1) ; 

LEVEL(q,i+l)  =  condition(f ) ; 

} 

free_funct(f ) ; 

thr  =  THRESH(q, nlev-1) ; 

lev  =  LEVEL(q, nlev-1); 
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return(  abaci (thr, lev, r_mean  +  r_inf)  ); 

} 


nain( ) 

{ 

extern  QUANTPTR  q_alloc(),  q; 
extern  FILB  *fopen(); 

extern  int  fclose( ) ,set_initial( ) , print_quantizer( ) , zero_ quant ( ) ; 
extern  int  saet_initial( ) ; 

extern  double  sqrt( ) , fabs( ) , find_quant( ) , sgn( ) ; 

extern  double  x_mean,n_mean,r_mean,x_inf ,n_inf ,r_inf; 

extern  double  x_std,x_var,n_std,n_var,r_std,r_var; 

extern  double  kl,k2; 

double  add, guess, diff, last_diff; 

double  nstd.xstd, scale; 

FILE  *io; 

QUANTPTR  qr; 
char  name [ 25 ] ; 
int  nlev.i.j; 

printf ("Minimum  Risk  Quantizer  (Abs  -  loss)  Program\n”); 

printfC’Enter  output  filename:”); 
gets (name) ; 

io  =  f open ( name,  V) ; 

printfC’Enter  number  of  quantizer  levels:  ”); 
scanf(M*d",8uilev) ; 

qr  =  q_alloc(nlev) ; 

nlev  /=  2; 

q  =  q_alloc(nlev) ; 

printf ("Bnter  signal  mean  and  std:  "); 
scanf(”*lf  %lf" ,&x_mean,&xstd) ; 
printfC’Enter  noise  mean  and  std:  ”); 
scanf("*lf  Xlf",&n_mean,&nstd) ; 

scale  =  xstd;  /*  Set  scale  to  calculate  on  N(mean,l.)  */ 

x_std  =1.0;  /*  then  rescale  {r},  {y}  back  out  later.*/ 
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n_std  =  nstd  /  scale; 


x_var  =  x_std  *  x_std; 
n_var  =  n_std  *  n_std; 
rvar  =  x_var  +  n_var ; 
r_std  =  sqrt(r_var); 
rmean  =  xmean  +  nmean; 

x_inf  =  5.0  *  x_std; 
n_inf  =  5.0  *  n_std; 
r_inf  =  5.0  *  r_std; 

/* - 

if(nlev  <  12) 
increment  =  0.1; 
else  if(nlev  <  45) 
increment  =  0.01; 
else  if(nlev  <  129) 
increment  =  0.001; 
else 

increment  =  5.0e-8; 

- - 

increment  =  INCREMENT; 
add  =  increment; 

guess  -  xmean  +3.  *  increment; 

kl  =  x_var  /  (x_var  +  n_var); 
k2  =  (xmean  *  nvar  -  nmean  *  x_var) 
/  (x_var  +  n_var); 

zero_quant ( q) ; 

sset_initial(q, guess) ; 

printf ( " Initial  Quantizer: \n" ) ; 

print_quantizer(q) ; 

printf ("\nNext:\n") ; 

diff  =  find_quant(q) ; 

print_quantizer(q) ; 

printf C’NnDiff:  Sllf\n”,diff) ; 

if  (fabs(diff)  >=  FIDELITY)  { 
lastdiff  =  diff; 
guess  +=  add; 
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set_initial(q, guess) ; 
diff  =  find_quant(q) ; 
print_quantizer(q) ; 
printf ("\n") ; 


Now  detenaine  if  we  are  going  in  the  right  direction 


if  (  diff/last_diff  >  1.  )  { 
guess  -=  2.  *  add; 
add  =  -add; 
last_diff  =  diff; 
set_initial(q, guess) ; 
diff  =  find_quant(q) ; 
print_quantizer(q) ; 
printf ("\n”) ; 


Now  we  just  keep  incrementing  guess  until  we  have  it  bracketed, 
determined  via  a  sign  change  in  diff. 


while  (  (sgn(diff)  ==  sgn(last_diff) ) 

&&  (fabs(diff)  >=  FIDELITY)  )  { 
last_diff  =  diff; 
guess  +=  add; 
set_initial(q, guess) ; 
diff  =  find_quant(q) ; 
printquantizer(q) ; 
printf(”\n") ; 


We  ve  bracketed  guess  now  and  only  monitor  sign  change  to 
determine  direction  of  add...  It  will  continuously  be 
divided  in  half  since  we  know  both  bracket  limits. 


while  (  (fabs(diff)  >=  FIDELITY)  &&  (fabs(add)  >=  FIDELITY)  ) 
add  /=  (sgn(diff)  !=  sgn(last_diff) )  ?  -2.  :  2.; 
last  diff  =  diff: 


guess  +=  add; 
set_initial(q, guess) ; 
diff  =  find_quant(q) ; 
print_quantizer(q) ; 
printf("\n") ; 

} 

} 

for(i=0;  i<nlev;  ++i) 

LKVEL(q,i)  =  LEVEL(q,i)  -  xmean ; 

j  -  nlev; 

for(i=0;  i<nlev;  ++i)  { 

THRESH(qr.i)  =  THRESH(q.O)  -  scale  *  (THRESH(q, j ) -THRESH (q,0) ) ; 
LEVEL(qr.i)  =  xmean  -  scale  *  LEVEL(q, j-1) ; 

— j; 

THRESH( qr , i+nlev )  =  THRESH(q,0)  +  scale  *  (THRESH(q, i)-THHESH(q, 0) ) ; 
LEVEL(qr, i+nlev)  =  x_mean  +  scale  *  LEVEL(q.i); 

} 

THRESH(qr.O)  =  -10000.; 
print_quantizer(qr) ; 

/*  Scale  to  actual  stdev  */ 

fprint_quant(qr, io) ; 
fprintf  ( io,  "*1  An" ,  10000 . ) ; 

free_quant(q) ; 
f  ree_quant ( qr ) ; 

fclose( io) ; 
exit(0) ; 

} 


Program  to  generate  data  for  Risk  Surfaces 


/* - 

♦include  <stdio.h> 

♦include  <math.h> 

♦include  <function.h> 

♦include  Quantizer. h> 

QUANTPTR  q,  qrisk,  qunif,  qmax; 
double  r_inf; 
double  x_inf; 
double  n_infj 

double  n_mean,n_std,x_mean, x_std, rjnean, r_std; 
double  kl,k2,x_var,n_var,r_var; 


*/ 


double  p_normal (mean, std, x) 
double  mean; 
double  std; 
double  x; 

{ 

extern  double  fabs( ) , exp() ; 
double  arg; 

if  (faba(x)  >  (5.0  *  std  +  mean)) 
return(0. ) ; 
else  { 

arg  =  -(x~mean)#(x-mean)/(2. *std*std) ; 
return(  0.3989422804  *  exp(arg)  /  std); 

} 

} 

double  p_noise(n) 
double  n; 

{ 

extern  double  n_mean,  n_stdf  p_normal(); 
return ( p_normal ( n_mean , n  std,n)); 

} 
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double  p_x(x) 
double  x; 

{ 

extern  double  xmean ,  x_std,  p_nonnal(); 
return(p_nonnal(xjnean,x_std,x) ) ; 

} 

/* - 

Observation  probability  density  function. 
Defined  as  the  convolution  of  p(x)  and  p(n)  at 
the  observation  point  r. 


double  p_r(f) 

FUNCTPTH  f; 

{ 

extern  double  p_noraal(),  r_mean,  r_std; 
double  r; 

r  =  FDPARM(f , 1) ; 

return ( p_normal ( r_mean , r_std, r) ) ; 

} 


double  dist(f) 

FUNCTPTR  f; 

{ 

double  r,y; 
extern  double  p_r(); 
r  =  FDPAflM( f , 1) ; 
y  =  FDPARM(f ,2) ; 
r  =  (r  -  y)  *  (r  -  y)  *  P_r(f); 
return(r) ; 

} 


-140- 


double  inner(f) 

FUNCTPTR  f; 

{ 

extern  double  p_noi.se ( )  ,p_x( ) ; 
double  x,y,r; 

x  =  FDPAHM(f.l); 
y  =  FDPARM( f , 2) ; 
r  =  FDPAHM( f , 3) ; 

return(  (x  -  y)  *  (x  -  y)  *  p_noise(r  -  x)  *  p_x(x)  ); 

} 


double  innerisk(f) 

FUNCTPTR  f; 

{ 

extern  double  p_r(); 

extern  double  x_var,n_var, r_var,kl,k2; 
double  value, r,y,c; 
r  =  FDPAHM(f , 1) ; 
y  =  FDPARM(f,2); 
c  =  (x_var  *  n_var)  /  r_var; 

value  =  (kl  *  r  +  k2  -  y)  *  (kl  *  r  +  k2  -  y); 

/*  printf("c:%lf  kl:*lf  k2:%lf  squared  termr%lf\n",c,kl,k2, value) 
*/ 

value  +=  c; 

return(value  *  p_r(f)); 

} 


double  calc_risk(q) 

QUANTPTR  q; 

( 

extern  double  innerisk() , integral () , r_inf , r_mean; 
double  rl, ru, y, risk, pos , neg, two_mean; 
int  i,num_levels; 
extern  FUNCTPTR  f_alloc(); 

FUNCTPTR  f; 


f  =  f_alloc(4) ; 
FFPARM(f,0)  =  innerisk; 
risk  =  0. ; 
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num_levels  =  NLEVELS(q); 


if ( (NLEVELS(q)  X  2)  ==  0)  {  /*  even  */ 

for(i  =  0;  i  <  num_levels  -  1;  ++i)  { 
y  =  LEVEL(q, i) ; 
rl  =  THRESH(q.i); 

FDPAflM( f , 2)  =  y; 
ru  =  THRESH(q, i+1) ; 
pos  =  integral (rl, ru, f) ; 
risk  +=  pos; 

} 

y  =  LEVEL(q,num_levels-l) ; 

FDPAHM(f , 2)  =  y; 
rl  =  THRESH(q,num_levels-l) ; 
ru  =  rjaean  +  rinf ; 
pos  =  integral (rl.ru, f) ; 
risk  +=  pos; 

} 

else 

printf( "Section  not  implemented  for  odd  number  of  levels\n"); 

free_funct(f) ; 

return (risk) ; 

} 


double  calc_mrisk(q) 

QUANTPTR  q; 

{ 

extern  double  dist( ) ,p_r( ), integral (), r_inf, r_mean; 

extern  double  x_var,n_var, r_var,kl,k2; 

double  rl, ru,y, c, d,p, risk, pos, neg, two_mean; 

int  i,num_levels; 

extern  FUNCTPTR  f_alloc(); 

FUNCTPTR  fl,  f2; 

fl  -  f_alloc(4); 
f2  =  f_alloc(4) ; 

FFPARM(fl.O)  =  p_r; 

FFPARM(f2,0)  =  dist; 
c  =  (x_var  *  n_var)  /  r_var; 
risk  =  0. ; 


num_levels  =  NLEVELS(q); 


if ( (NLEVELS(q)  X  2)  ==  0)  {  /*  even  */ 

for(i  =0;  i  <  num_levels  -  1;  ++i)  { 
y  =  LEVEL(q.i); 
rl  =  THRESH(q, i) ; 

FDPAHM(f2,2)  =  (y  -  k2)  /  kl;  /*  convert  to  max  level 

*/ 

ru  =  THHESH(q, i+1) ; 
p  =  integral (rl.ru, fl) ; 
d  =  integral ( r 1, ru, f2) ; 
oos  =  c  *  p  +  kl  *  kl  *  d; 

/*  printf("c:Xlf  rl.'Xlf  ru.'Xlf  p:Xlf  kl:\lf  d:*lAn",c,rl,ru,p,kl,d) ; 
*/ 

risk  +=  pos; 

} 

y  =  LEVEL(q,num_levels-l) ; 

FDPAHM(f2,2)  =  (y  -  k2)  /  kl;  /*  convert  to  max  level  */ 

rl  =  THRESH(q,num_levels-l) ; 
ru  =  rjnean  +  r_inf ; 
p  =  integral (rl,ru, fl) ; 
d  =  integral(rl,ru,f2) ; 
pos  =  c  *  p  +  kl  *  kl  *  d; 

/*  printf("c:Xlf  rl:Xlf  ru:Xlf  p:%lf  kl:*lf  d: *lAn" , c,  rl,  ru.p.kl, d) ; 
*/ 

risk  +=  pos; 

} 

else 

printf( "Section  not  implemented  for  odd  number  of  levels\n"); 

free_funct( fl) ; 
free_funct(f2) ; 
return (risk) ; 

} 


double  db(x) 
double  x; 

{ 

extern  double  logl0(); 
return(10.  *  loglO(x)); 

} 
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double  dbinv(x) 
double  x; 

{ 

extern  double  pow(); 

return(  pow(10.,  0.1  *  x)  ); 

} 


main( ) 

{ 

extern  QUANTPTR  q_alloc(),  q,  qrisk,  qunif,  qmax; 
extern  double  p_r( ) ; 

extern  double  r_var,  kl,  k2,  x_inf,  n_inf,  r_inf; 
extern  double  fabs(),  loglOO,  pow(),  sqrt(); 
extern  double  calc_riak( ) ,calc_mrisk( ) , db( ) , dbinv( ) ; 
extern  int  free_quant( ) ; 

extern  double  x_mean,x_std,n_mean,n_std, r_mean, r_std; 
double  thresh[128] , level [128) , delta; 
extern  double  x_var,n_var; 

FILE  *io,  *maxfile; 

double  area,  laat_area,  (*p)  (); 

double  sn_ratio,ri3k; 

double  rlower.rupper, incr.nstd; 

double  xmlower.xmupper,  xmincr,  run  lower,  nmupper,nmincr; 
int  i, j,num, numlevels; 

printf ("Risk  Curve  Generation  Program\n"); 
io  =  f open ( "RISKPLOT  OUT  B","w"); 
maxfile  =  fopen("MAX  DATA  B","r"); 

P  =  P_r; 

printf ("Enter  range  and  increment  of  S/N  ratio. \n"); 
printf  ( "(-10dJ3  to  15dB  by  5dB  — >  -10.  15.  5.)\n"); 
scanf("%lf  Slf  %lf",&rlower,&rupper,&incr) ; 

printf ( "Enter  range  and  increment  of  signal  mean:  "); 
scanf("%lf  %lf  %lf"  ,  Sixmlower,  &xmupper ,  Stxmincr) ; 
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if(xmincr  <=  0.)  xmincr  =  1.; 


printf ("Enter  range  and  increment  of  noise  mean:  "); 
scanf("Xlf  *lf  *lf",&nmlower,&nmupper,&nmincr) ; 

if(nmincr  <=  0.)  nmincr  =  1.; 

printf ("Enter  number  of  levels:  "); 
scanf ( "%d" , &num ) ; 


for(i=2;  i<=num;  ++i) 
for( j=0;  j<i;  ++j) 

fscanf  (maxfile,  "Xlf  *lf" ,&thresh[ j]  , &level[j] ) ; 

fclose(maxfile) ; 

for( j=num-l;  j>0;  — j) 
thresh[ j]  =  thresh[j-l]; 

thresh[0]  =  -INFINITY; 
thresh [num]  =  INFINITY; 
level [num]  =  0.; 

x_std  =  1.0; 

qmax  =  q_alloc(num) ;  /*  Max-Lloyd  Std.  Normal  */ 

for(i  =0;  i  <  num;  i++)  {  /*  risk  surf  only  makes  sense  for 

THRESH (qmax, i)  =  thresh[i];  /*  x_mean  =  0  */ 

LEVEL (qmax, i)  =  level(i]; 

} 


nmean  =  nmlower; 
while(n_mean  <=  nmupper)  { 
x_mean  =  xmlower; 
while(x_mean  <=  xmupper)  { 
nstd  =  r lower; 

while(nstd  <=  rupper)  { 
n_std  =  sqrt(l.  /  dbinv(nstd) ) ; 
r_mean  =  x_mean  +  nmean ; 
r_var  =  x_std  *  x_std  +  n_std  *  nstd; 
r_std  =  sqrt(r_var); 
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r_inf  =  5.0  *  r_std; 
x_inf  =  5.0  *  x_std; 
n_inf  =  5.0  *  n_std; 

q  =  q_alloc(num) ; 
qrisk  -  q_alloc(num) ; 
qunif  =  q_alloc(num) ; 

x_var  =  x_std  *  x_std; 
n_var  =  n_atd  *  n_std; 

kl  =  x_var  /  (x_var  +  n_var); 

k2  =  (x_mean  *  n_var  -  n_mean  *  x_var)  /  (x_var  +  n_var) ; 

delta  =  2.  *  r_inf  /  num; 

THRESH ( qunif, 0)  =  2.  *  r  tnean  -  r_inf; 

LEVEL(qunif.O)  =  THRESH( qunif ,0)  +  delta  /  2.; 

for(i  =  1;  i  <  num;  ++i)  { 

THRESH(qunif.i)  =  THRESH( qunif , i-1)  +  delta; 

LEVEL( qunif, i)  =  LBVEL(qunif , i-1)  +  delta; 

} 

for(i  =  0;  i  <  num;  ++i)  [ 

THHESH(q,i)  =  r_std  *  threah[i]  +  r_mean; 

LEVEL(q,i)  =  r_std  *  level[i]  +  r_mean; 

THRESH (qrisk, i)  =  THRESH(q.i); 

LEVEL (qrisk, i)  =  kl  *  LEVEL(q,i)  +  k2; 

} 

THRESH (qmax,0)  =  r_mean  -  r_inf; 

THRESH(q,0)  =  r_mean  -  r_inf; 

THRESH ( qrisk, 0)  =  THRESH(q, 0) ; 

sn_ratio  =  db(x_var/n_var) ; 

fprintf ( io, "Xlf  Xlf  Stlf  Xlf",x_mean,x_std,n_mean,n_std) ; 
printf("Xlf  Xlf  Xlf  Xlf",x_mean,x_std,n_mean,n_std) ; 

printf ("Max-Lloyd  Std.  Normal: \n\n") ; 

print_quantizer(qmax) ; 
printf ("\n") ; 
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risk  =  calc_risk(qmax) ; 

fprintf(io, "  *lf",risk); 
printf("  Xlf",risk); 

printf (”Max-Lloyd  Quantizer: \n\n") ; 

print_quantizer(q) ; 
printf("\n") ; 
risk  =  calc_risk(q) ; 

fprintf(io,”  Xlf'.risk); 
printf (”  Xlf'.risk); 

/*  risk  =  calc_mrisk(q) ; 

fprintf(io,"  Xlf”,  risk) ; 
printf ("  Xlf",risk); 


printf  ( "\n\nMinimum-Risk  Quantizer: \n\n”) ; 

print_quantizer(qrisk) ; 
printf ("\n") ; 
risk  =  calc_risk(qrisk) ; 
printf ("\n"); 

fprintf(io,''  Xlf”,risk); 

printf (n  %lf\n",risk); 

risk  =  calcjnrisk(qrisk) ; 
fprintf(io,"  Xlf”,risk); 
print fC'mrisk:  Xlf\n" , risk) ; 

calc_risk  and  calcjnrisk  produced  exactly  same  values  - */ 

printf (”\n\nUnifon»  Quantizer: \n\n") ; 
print_quantizer(qunif ) ; 
printf ("\n") ; 
risk  =  calc_risk(qunif ) ; 
printf("\n") ; 

fprintf(io,"  Xlf\n",risk) ; 
printf("  Xlf\n",risk); 
free_quant(q) ; 
free_quant(qrisk) ; 
free_quant(qunif ) ; 
nstd  +=  incr; 

} 

x_mean  +=  xmincr; 

} 

n_mean  +=  nmincr; 


/* 

*/ 

/* - 
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} 

fclose(io) ; 
exit(O) ; 
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